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ABSTRACT 


This  study  examined  follow-up  achievement  performance  and  expressed 
attitudes  toward  STAT1,  the  University  of  Alberta  computer- assisted 
instruction  statistics  program  offered  to  students  at  the  graduate 
level.  An  opinionaire  and  an  achievement  test  designed  by  the  author 
were  administered  to  persons  who  completed  the  program  in  Winter  Session 
1976-77,  Summer  Session  1977,  Spring  Session  1978  and  Summer  Session  1978. 
Multiple  matrix  sampling  procedures  were  used  in  developing  the  tests 
used  to  measure  achievement. 

Pearson  correlations  were  calculated  between  all  pairs  of  variables 
but  only  those  variables  dealing  with  expressed  liking  for  learning 
statistics  via  CAI,  and  perceived  usefulness  of  the  material  learned 
were  examined  in  detail. 

Results  indicated  that  achievement  levels  for  all  groups  were 
approximately  equal  and  tended  to  be  15-20%  below  final  course  grades. 
Positive  attitudes  toward  learning  via  CAI  were  maintained  by  all  groups 
despite  low  reported  use  of  the  knowledge  gained.  The  effects  of  such 
things  as  previous  knowledge  of  statistics  and  apprehension  about  taking 
the  program  were  discussed  as  influencing  factors . 

Recommendations  for  changes  in  STAT1  were  made  based  on  the  comments 
offered  by  participants  in  the  study  and  on  the  personal  experience  of 
the  author. 
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Chapter  1 
Introduction 

i 

Statement  and  Importance  of  the  Problem 

This  study  was  undertaken  for  the  purpose  of  evaluating,  on  a 
follow-up  basis,  a  computer -assisted  instruction  program  designed  to 
teach  statistics  to  graduate  students  in  Educational  Psychology  at 
the  University  of  Alberta.  As  such  it  comprises  about  75-801.  of 
the  course  hours  in  Educational  Psychology  502  (EdPsy  502) ,  the 
only  other  instruction  being  16  hours  of  lecture  time  devoted  to 
topics  in  research  design. 

The  specific  intent  of  the  study  was  to  provide  information  about 
the  retrospective  perceptions  of  students  who  had  completed  the  pro¬ 
gram  (known  as  STATl)  and  to  measure  their  performance  in  statistics 
six  months  to  two  years  after  completion  of  the  program. 

Upon  completion  of  STATl,  all  students  responded  to  a  question¬ 
naire  asking  them  to  rate  their  agreement  with  statements  about  per¬ 
ceptions  of  STATl.  These  statements  included  such  factors  as  difficulty 
of  the  content,  how  enjoyable  it  was  to  learn  via  computer- ass is ted 
instruction,  how  interesting  the  material  was  perceived  to  be,  and  how 
useful  the  program  was  in  helping  the  student  to  learn  statistics . 

For  the  purpose  of  this  follow-up  evaluation,  attention  was 
focussed  on  student  perceptions  as  they  related  to  two  main  areas: 
expressed  liking  for  learning  via  computer- ass  is ted  instruction,  and 
perceived  usefulness  of  the  material  learned.  Students  were  asked  to 
complete  a  questionnaire  containing  some  of  the  same  items  they  had 
rated  when  they  had  just  completed  the  program  so  that  post  and  follow¬ 
up  comparisons  could  be  made.  In  addition,  students  were  asked  to  rate 
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their  initial  apprehension  about  taking  EdPsy  502,  the  reasons  for 
this  apprehension,  and  also  to  rate  the  amount  and  type  of  use  they 
had  made  of  the  material  learned.  Students  were  invited  to  comment 
on  the  advantages  and  disadvantages  of  STAT1  and  to  make  suggestions 
for  additions  or  deletions  which  might  improve  the  program. 

The  study  was  done  for  a  number  of  reasons .  First ,  the  STAT1 
program  had  been  in  use  with  few  changes  since  1976  so  that  enough 
time  had  elapsed  to  make  a  follow-up  study  feasible.  Second,  it  is 
valuable  for  the  course  developers  to  know  if  a  program  is  meeting 
the  perceived  needs  of  its  consumers  and  fulfilling  the  objectives  it 
was  designed  to  satisfy.  Finally,  the  computer-assisted  instruction 
facility  at  the  University  of  Alberta  will  be  undergoing  changes  in 
1980  when  the  computer  equipment  now  in  use  is  recalled  by  the  manu¬ 
facturer  because  it  is  considered  obsolete.  Decisions  must  be  made 
about  the  nature  and  extent  of  service  to  be  provided  in  the  future 
not  only  by  the  facility  itself  but  by  the  individual  programs  cur¬ 
rently  available.  This  study  provides  information  about  how  well 
STAT1  is  meeting  the  needs  of  the  students  and  offers  suggestions  for 
changes  to  better  meet  those  needs . 

Limitations  of  the  Study 

No  attempt  was  made  to  compare  groups  of  students  who  had  taken 
STAT1  with  students  who  had  learned  statistics  through  classroom  in¬ 
struction  only  since  the  purpose  of  the  study  was  not  to  defend  computer- 
assisted  instruction  as  a  '’better”  means  of  instruction,  but  rather  to 
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gather  information  for  evaluating  STAT1  in  the  context  in  which  it 
was  used.  For  the  same  reason,  students  who  received  both  classroom 

/ 

instruction  (in  addition  to  research  design)  and  STAT1  instruction 
were  not  included. 

As  an  evaluation  study,  the  scope  of  generalizability  is  limited 
to  the  particular  program  (STAT1)  and  the  particular  type  of  student 
involved  (graduate  students  in  the  Faculty  of  Education) .  There  is 
no  evidence  that  findings  may  be  generalized  further  to  include  other 
course  contents,  other  levels  of  students  or  students  in  other 
universities . 

Finally,  the  study  was  designed  to  evaluate  the  STAT1  program  not 
the  particular  students  who  used  it.  Therefore,  only  group  means  were 
used  to  demonstrate  achievement.  Achievement  test  scores  may  give  an 
indication  as  to  how  the  group  has  maintained  knowledge  of  statistics 
but  may  not  be  used  for  decision  making  regarding  individuals .  This 
is  because  the  method  of  data  collection  involved  the  use  of  multiple 
matrix  sampling  (Shoemaker,  1970a)  so  that  all  subjects  did  not  respond 
to  the  same  items. 

Definition  of  Terms 


1.  STAT1  -  the  coirputer- assisted  instruction  program  to  teach 
statistics  to  graduate  students  at  the  University  of  Alberta.  It  includes 
the  instructional  component  as  well  as  laboratory  assignments  associated 
with  the  program. 
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2.  EdPsy  502  -  the  course  name  applied  to  a  course  in  statistics 
offered  to  graduate  students  by  the  department  of  Educational  Psychology 
at  the  University  of  Alberta.  It  includes  STAT1  and  16  hours  of  class¬ 
room  lectures  on  research  design  topics. 

3.  Computer- as sis ted  instruction  -  a  linear  program  operating 

in  the  tutorial  mode,  designed  to  replace  classroom  instruction  of  the 
specific  content;  in  this  case,  statistics. 

4.  Attitude  -  "the  bearing  assumed  by  a  person  or  body  of  persons 
indicative  of  feeling,  opinion,  etc."  (Webster,  1960,  p.63).  Attitude 
shall  be  understood  to  refer  to  feelings  or  opinions  held  by  students 
about  STAT1  as  a  result  of  their  experience  with  the  program. 


Chapter  2 

Survey  of  the  Literature 


What  is  CAI? 

The  use  of  computers  in  education  takes  many  forms,  ranging  from 
administrative  functions  such  as  time-tabling  to  teaching  functions 
such  as  drill- and-practice  routines  and  testing.  A  wide  variety  of 
these  latter  functions  are  included  in  the  term  Computer-Based 
Education  (CBE) .  Unfortunately,  when  one  tries  to  define  or 
categorize  more  specifically,  the  proliferation  of  terminology  in 
use  is  confusing.  To  demonstrate  this  point,  the  reader  is  directed 
to  Salisbury  (1971)  who  lists  21  terms  found  in  the  literature  to  be 
used  interchangeably,  although  presumably  not  intended  to  mean  precisely 
the  same  thing.  Examples  include  such  terms  as  computer-based  in¬ 
struction,  computer- assisted  instruction,  computer- aided  learning,  and 
computer- managed  instruction. 

If  one  considers  the  student  to  be  the  main  focus  of  attention 
in  the  field  of  education  (a  risky  assumption ,  perhaps)  then  computer 
use  may  be  defined  in  terms  of  student  interaction.  Two  terms  seem 
to  lend  clarity  to  the  problem:  1)  computer-managed  instruction  (CMI), 
and  2)  computer-assisted  instruction  (CAI) .  Computer-managed  instruc¬ 
tion  is  generally  agreed  to  include  such  functions  as  monitoring  student 
progress  on  assignments,  selection  and  assignment  of  appropriate  learning 
experiences,  instructional  development,  and  record  keeping  (McLagan  § 
Sandborgh,  1977;  Smolinski,  1977).  Student  interaction  is  minimal  and 
may  consist  solely  of  submitting  responses  to  assignments  into  the  computer 
for  analysis  and  marking  (Dixon  §  Judd,  1977).  Hence  Oil  will  not  be 
considered  further  in  this  study. 
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Computer- assisted  instruction  (CAI)  is  defined  as  "...the  use  of 
the  computer  for  direct  instruction  of  students.  Computer  assisted 

/ 

instruction  covers  such  uses  as  drill -and-practice,  problem  solving, 
simulation,  and  tutoring,  but  does  not  include  computer -managed 
instruction  or  the  teaching  of  computer  science."  (Kulik  §  Jaksa,  1977, 
p.  15).  From  the  broad  and  general  discriptions  of  CAI  found  in  the 
literature,  little  is  to  be  learned.  Where  explicit  definitions  are 
given,  there  seems  to  be  agreement  on  these  points: 

1)  instructional  material  is  presented  to  the  learner  via 
the  computer,  and 

2)  interaction  occurs  between  the  student  and  the  computer 
(Salisbury,  1971) . 

Having  looked  at  this  definition  of  CAI ,  it  seems  to  make  little 
difference  if  one  substitutes  the  word  "aided"  for  "assisted"  or  the 
word  "teaching"  for  "instruction".  There  is,  however,  a  difference 
between  "learning"  and  "instruction".  As  Kearsley  (1978)  has  pointed 
out,  computer-assisted  learning  (CAL)  focuses  on  the  computer  as  a  tool 
of  the  student,  while  computer- assisted  instruction  focuses  on  the 
computer  as  a  tool  of  the  instructor.  CAL  involves  the  issue  of  learner 
control  which  is  not  a  concern  of  this  particular  work. 

Within  the  concept  of  CAI,  as  defined  above,  there  exist  a 
number  of  instructional  modes:  drill -and-practice ,  simulation, 
tutorial,  calculation,  inquiry  (similar  but  not  identical  to  in¬ 
formation  retrieval),  modelling,  instructional  games,  and  Socratic 
inquiry  (McLagan  $  Sandborgh,  1977).  These  may  be  used  singly  or 
several  combined  into  one  CAI  program.  Those  most  commonly  used,  in 
terms  of  number  of  programs  available,  are  drill-and-practice,  tutor- 
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ial,  problem  solving,  simulation,  and  a  mixture  of  modes  (Kearsley, 
1976a) . 

/ 

With  a  definition  of  CAI  and  a  list  of  the  common  instructional 
modes,  it  should  be  a  relatively  simple  matter  to  evaluate  reported 
studies  employing  CAI,  but  it  is  not.  Very  little  effort  has  been 
made  by  various  authors  (e.g.  Borman,  1970;  Doty  §  Doty,  1964)  to 
define  and  outline  the  programs  under  investigation.  It  is  often  not 
clear  whether  the  program  represents  a  whole  course  (i.e.  the  bulk  of 
instruction  in  a  subject)  or  only  a  part.  The  problem  is  such  that 
Seidel  et  al  (1978)  have  advocated  dropping  the  term  CAI  in  favor  of 
project- specific  terms  which  describe  the  ways  in  which  the  computer 
is  being  used.  Some  uniform  definition  and  usage  of  terminology  is 
certainly  needed. 

Why  use  CAI? 

Having  determined  what  we  mean  by  CAI,  it  is  now  important  to 
look  at  the  reasons  for  using  it  as  a  method  of  instruction.  The  most 
comp rehens ive  list  of  computer  capabilities  has  been  prepared  by  Ball 
(1971),  as  follows: 

1)  ’’...the  power  to  record,  manipulate,  remember  and  respond 
to  many  persons  individually  and  simultaneously.”  (p.  187). 

2)  adaptable  to  an  unlimited  number  of  variations 

3)  reacts  only  to  the  input  of  the  student 

4)  displays  infinite  patience  and  lack  of  emotionality 

5)  gives  and  receives  feedback  immediately 

6)  identifies  individual  difficulties 


. 
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7)  allows  progress  at  a  self-determined  rate,  including  test 
taking 

i 

8)  frees  the  instructor  to  attend  to  the  needs  of  the  individual 
student  with  problems 

9)  requires  less  time  to  reach  educational  objectives 

10)  improves  retention. 

The  first  8  capabilities  are  properties  of  most  computer  systems 
or  are  programmed  into  the  instructional  package.  The  last  2  will  be 
discussed  later.  The  listed  capabilities  are  somewhat  vague  and  as 
such  allow  equally  for  very  good  and  very  poor  program  design.  The 
quality  depends  very  much  on  the  program  author. 

Hess  and  Tenezakis  (1970)  have  taken  a  slightly  different  approach, 
looking  at  the  computer  as  a  socializing  agent,  and  attempting  to  refute 
claims  that  learning  via  CAI  is  ’’dehumanizing”.  Their  list  of  the 
properties  of  CAI  includes: 

1)  capability  for  interaction 

2)  acts  as  a  reinforcing  agent 

3)  uses  humanizing  techniques  such  as  addressing  students  by  name 

4)  has  intrinsic  interest  as  a  motivating  and  engaging  feature  - 
this  is  observed  especially  in  studies  involving  children. 

Finally,  in  a  more  recent  attempt  to  dispel  the  belief  that  CAI  is 
dehumanizing,  Magidson  (1977)  has  offered  the  following: 

1)  CAI  lessons  are  not  impersonal  -  stresses  patience  and  immediate 
feedback. 

2)  CAI  lessons  are  an  extension  of  a  human  author  -  the  computer 
does  not  create. 

3)  Drill  and  practice  is  no  more  dehumanizing  than  classroom  drills. 


‘ 
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4)  CAI  does  not  necessarily  promote  rote  learning  -  it  makes  use 
of  tutorial  and  simulation  modes  for  higher  level  functioning. 

5)  CAI  curriculum  does  not  have  to  be  fixed  -  changes  are  easily 
made . 

6)  Students  report  that  CAI  learning  is  enjoyable. 

7)  Some  CAI  systems  include  a  personalized  communication  system 
(e.g.,  PLATO). 

Rather  than  being  a  rigid,  depersonalized  instructor,  the  computer 
appears  to  be  a  faster,  more  efficient  instructor  with  reinforcing  and 
record-keeping  capabilities  beyond  the  reach  of  the  most  fastidious  human 
teacher.  It  will  be  noted,  however,  that  most  of  the  computer  capabilities 
listed  are  programming  variables,  and  hence  are  only  as  good  as  the  course 
designer.  This  substantial  human  element  may  make  the  difference  between 
an  interesting  and  effective  program  for  learning  and  an  ineffective  one. 

What  does  all  this  mean  for  the  student?  The  implication  has  been 
that  the  student  can  be  expected  to  learn  at  least  as  well  as  in  a 
traditional  instruction  mode,  to  experience  considerable  time  savings, 
and  to  express  generally  positive  attitudes  toward  CAI  as  a  mode  of 
instruction  (Jamison,  Suppes  §  W[ells,  1974;  Petruk,  1978).  The  literature 
evaluating  CAI  as  a  means  of  instruction  will  be  examined  as  it  relates 
to  each  of  these  three  considerations,  i.e.,  effectiveness  of  CAI  related 
to  achievement,  attitudes  toward  CAI,  and  time  savings  using  CAI. 

Effects  of  CAI  on  Achievement 


As  Kearsley  (1977)  has  pointed  out,  it  is  not  enough  to  say  that  CAI 
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is  effective  or  ineffective  as  a  teaching  mode.  Effectiveness  needs  to 
be  examined  in  relation  to  many  variables,  such  as  learning  styles, 
personality  characteristics,  and  subject  matter  versus  type  of  in¬ 
struction.  The  typical  report  on  the  effectiveness  of  CAI  involves  a 
comparison  of  achievement  scores  for  students  receiving  instruction  via 
CAI  with  those  of  students  receiving  traditional  instruction.  Bundy 
(1964)  and  Fletcher,  Suppes  and  Jamison  (1972)  report  extensively  on 
studies  of  this  type.  The  reported  results  almost  always  favor  CAI  over 
traditional  instruction,  citing  higher  test  results  or  improved  function¬ 
ing  in  terms  of  grade  level  (elementary  students) .  The  point  which  is 
often  overlooked  is  that  instruction  via  CAI  usually  consists  of  drill- 
and-practice  sessions  only.  Thus,  such  studies  are  more  likely  reporting 

on  the  effectiveness  of  the  drill -and-practice  routine  in  reinforcing 
mathematical  principles  learned  in  the  classroom,  than  on  the  effect¬ 
iveness  of  CAI  as  the  principle  mode  of  instruction.  It  is  difficult 
to  be  certain  of  this,  however,  since  the  details  of  the  program  under 
investigation  are  often  omitted. 

Edwards  et  al  (1975)  reviewed  the  literature  on  effectiveness  of 
CAI.  Their  findings  were  as  follows: 

1)  Traditional  instruction  supplemented  by  CAI  was  always  more 
effective  than  traditional  instruction  alone.  CAI  usually 
referred  to  drill- and-practice . 

2)  When  CAI  was  substituted  for  all  or  part  of  the  traditional 
instruction  9  studies  showed  increased  achievement  scores  with 
CAI  while  8  showed  no  difference. 

3)  When  different  CAI  modes  were  compared,  none  was  consistently 
more  effective  than  the  others. 
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4)  There  was  no  difference  between  the  effectiveness  of  CAI 
and  other  non- traditional  methods  such  as  tutoring  or 

i 

language  labs. 

5)  In  studies  comparing  retention  with  CAI  and  traditional 
instruction,  2  out  of  3  provided  some  evidence  that 
retention  was  less  with  CAI. 

Some  evidence  also  exists  which  suggests  that  CAI  may  be  more  effective 
for  students  who  are  economically,  intellectually,  and/or  physically 
disadvantaged  than  for  students  who  are  motivated  and  oriented  towards 
school  achievement  (Jamison,  Suppes  §  Wells,  1974;  Kearsley,  1977; 

Suppes  $  Momingstar,  1969) .  Morgan  (1975)  and  Fletcher  and  Beard  (1973) 
report  improved  performance  of  hearing  impaired  students  especially  in 
mathematics,  using  CAI  drill  and  practice  to  supplement  classroom  in¬ 
struction. 

It  is  rare  to  find  a  study  in  which  CAI  was  used  for  the  total 
instruction  of  a  course,  but  Petruk  (1978)  has  reported  such  a  study. 

First  year  electrical  apprentices  were  divided  into  experimental  (CAI) 
and  control  (classroom  instruction)  groups  for  courses  in  electrical 
theory  and  electrical  mathematics.  The  experimental  group  received  all 
instruction  in  these  courses  via  CAI.  When  achievement  scores  were 
compared  on  common  exams,  no  differences  were  found  between  control  and 
experimental  groups . 

The  evidence  presented  is  less  than  conclusive.  Generally,  so 
little  information  is  reported  that  it  is  impossible  to  know  if  the 
findings  are  comparable  across  studies.  The  question  of  retention  remains 
virtually  unanswered  and  one  can  only  reasonably  conclude  that  students 
learn  via  CAI  at  least  as  well  as  by  classroom  instruction  alone. 
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Attitudes  towards  CAI 


Another  concern  which  might  affect  student  performance  when  learning 
via  CAI  is  the  attitude  of  the  student  toward  the  instructional  method. 
Problems  arose  again  when  comparing  findings  across  studies  in  that  some 
reported  attitudes  toward  CAI  as  the  sole  mode  of  instruction,  while 
others  were  looking  at  a  more  restricted  use  of  CAI,  the  details  of  which 
were  often  lacking. 

In  the  overall  study  of  CAI,  relatively  few  studies  have  examined 
attitudes  toward  learning  via  CAI,  and  most  of  those  which  have  use 
experimenter  constructed  tests  of  unknown  reliabilities  (King,  1975)  . 
Generally,  the  programs  under  study  are  of  short  duration  (less  than 
10  hours)  and  undoubtedly  influenced  by  novelty  effects  of  unknown 
quantity.  Nonetheless,  research  findings,  while  inconclusive,  are  de¬ 
serving  of  attention.- 

Hartley  (1966),  in  reviewing  work  done  in  the  early  1960's,  found 
conflicting  evidence.  He  lists  9  studies  which  reported  favourable 
attitudes  toward  instruction  via  CAI,  and  5  in  which  neutral  attitudes 
were  found.  Students  were  reported  to  have  a  preference  for  working 
with  both  teacher  and  machine  rather  than  either  one  alone.  Mitzel 
(1967)  reported  mostly  favourable  attitudes  although  students  stated 
they  missed  classroom  discussion.  High  school  chemistry  students  felt 
that  CAI  was  superior  to  traditional  instruction  but  still  preferred 
the  traditional  mode  because  "the  teacher  has  a  personality"  (Summerlin, 
1971) . 

In  her  extensive  review  of  the  impact  of  CAI  on  student  attitudes, 
King  (1975)  demonstrated  the  lack  of  definitive  findings  by  showing  that 
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while  no  study  reported  a  decrease  in  attitude  favorability  following  CAI 

use,  some  reported  increased  favorability  and  others  reported  no  change. 

/ 

Finally,  Lower  (1976)  cited  two  comments  frequently  made  by  students, 
namely  that  computer  learning  is  no  less  impersonal  than  large  lecture 
halls,  and  that  computer  learning  allows  one  to  make  mistakes  in  private. 

Some  authors  have  attempted  to  determine  which  factors  were  more 
likely  to  be  associated  with  positive  attitudes  toward  computer  based 
learning.  However,  the  results  were  often  contradictory.  For  example, 
it  was  reported  that  positive  attitudes  are:  1)  not  related  to  achieve¬ 
ment  (Borman,  1970;  Eigen,  1963)  or  2)  related  to  achievement  for  females 
but  not  for  males  (Doty  §  Doty,  1964);  3)  dependent  upon  aptitude  (Mitzel, 
1967)  or  4)  independent  of  aptitude  (Broderick  §  Lovatt,  1975);  5)  depend¬ 
ent  upon  the  time  taken  to  complete  the  CAI  course  (Mitzel,  1967)  or 
6)  independent  of  time  taken  to  complete  the  course  (Borman,  1970) .  Some 
work  has  begun  researching  personality  variables  which  are  related  to 
positive  attitudes  toward  CAI  but  it  is  also  inconclusive  as  yet.  A 
statement  from  Levien  (1972)  sums  up  the  situation: 

In  any  event,  the  evidence  is  clearly  not  consistent  with 
the  hypothesis  that  students  hold  fundamentally  hostile 
attitudes  toward  instructional  computer  use.  Their  attitudes 
are  more  likely  to  be  shaped  by  their  perception  of  the  qual¬ 
ity  of  each  specific  application,  (p.  541) 

Levien's  opinion  that  quality  of  application  affects  student  attitudes 
toward  CAI  may  be  a  valid  one,  although  he  offered  no  support  for  it.  It 
is  worthy  of  note,  I  believe,  that  the  quality  of  a  CAI  program  is  virtually 
never  discussed  in  writing.  One  might  surmise  that  evaluations  based  on 
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poor  quality  programs  would  reveal  poor  student  performance  and  negative 
attitudes  toward  learning  via  CAI .  Osin  (1978)  has,  however,  not  found 

i 

this  to  be  true.  In  his  candid  paper  he  has  stated  that  CAI  programs 
in  his  setting  are  often  prepared  by  graduate  students  as  projects. 

As  such  he  felt  most  of  them  lacked  the  qualities  of  good  instructional 
design  and  smooth  operation  associated  with  sophisticated  answer 
analysis,  branching,  and  record  keeping.  Despite  this,  he  has  found 
that  students  using  these  CAI  programs  rated  CAI  highly  as  a  means  of 
learning  and  achieve  at  a  high  level.  This  he  has  dubbed  "The  Survival 
Principle"  -  students  will  succeed  despite  poor  teaching,  either  by 
teacher  or  terminal,  by  making  extra  efforts  outside  class.  Osin's 
failure  to  support  his  hypothesis  with  data  does  not  negate  it  as  an 
alternate  explanation,  although  one  would  hope  for  empirical  support  in 
the  near  future. 

In  all  cases  cited  in  this  section  thus  far,  the  CAI  programs  used 
represent  only  a  portion  of  the  instruction  in  a  particular  course.  Ex¬ 
trapolation  of  findings  to  the  case  where  learning  via  CAI  represents 
the  total  instruction  for  the  course  is  not  justified.  For  one  thing, 
the  length  of  the  program  in  most  cases  was  less  than  10  hours.  With 
such  short  programs,  the  students  may  view  it  as  "time  out"  from  regular 
classroom  lectures,  rather  than  a  desirable  mode  of  total  instruction. 
The  possibility  of  novelty  effects  has  already  been  mentioned. 

The  University  of  Calgary  has  developed  a  program  to  teach  degree¬ 
seeking  nurses  via  CAI  (Hannah,  1978) .  Initial  reaction  has  been  favor¬ 
able  and  work  is  continuing.  A  recent  report  by  Petruk  (1978)  involving 
electrical  apprentices  taking  two  courses  entirely  by  CAI  also  noted 
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generally  positive  attitudes  expressed  by  the  students.  Finally,  Alderman, 
Appel  and  Murphy  (1978)  have  compared  the  PLATO  and  TICCIT  systems  in  use 

i 

in  community  colleges.  On  the  PLATO  system,  where  less  than  one  third 
of  student  instruction  in  a  course  was  done  via  CAI,  attitudes  toward 
CAI  were  very  favourable  and  88%  of  students  denied  that  the  system 
was  boring  or  dehumanizing.  On  the  TICCIT  system,  where  courses  were 
taught  primarily  by  computer,  attitudes  were  favourable  but  less  so  than 
with  PLATO.  Students  stated  that  they  missed  the  interaction  with  the 
instructor.  While  not  conclusive,  there  is  some  suggestion  that  CAI 
alone  is  less  desirable  for  the  student  than  both  computer  and  instructor 
together.  (It  is  worthy  of  note  that  the  completion  rates  with  TICCIT  are 
are  very  low.  For  example  in  mathematics  a  16%  completion  rate  with 
TICCIT  is  contrasted  with  a  50%  completion  rate  with  the  lecture  method.) 


Time  Savings  with  CAI 

There  is  very  little  to  be  said  in  this  area  as  it  is  the  one  fact  on 
which  there  is  almost  complete  agreement  -  there  are  substantial  time 
savings  to  be  realized  using  CAI  over  conventional  methods  of  instruction 
(Edwards  et  al,  1975;  Grubb  $  Selfridge,  1964;  Jamison  et  al,  1974;  Petruk, 
1978;  Summerlin,  1971).  Of  course,  this  generally  refers  to  savings  for 
the  student.  If  the  author  of  a  program  is  also  course  instructor,  time 
not  spent  in  class  is  usually  more  than  made  up  for  by  time  spent  author 
ing  the  course.  Hunka  et  al,  (1976)  estimated  about  35-40  hours  of  author 
time  to  produce  1  hour  of  student  time.  Over  a  long  run  of  usage,  how¬ 
ever,  even  the  author/instructor  may  realize  a  time  saving.  Those  who 
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question  the  time  savings  for  the  student  (Osin,  1978)  do  so  on  the  basis 
that  no  measures  are  made  of  out-of-class  time  spent  on  the  course.  The 
same  is  true,  however,  for  students  in  conventional  classrooms,  so  that 
at  this  time,  this  issue  remains  unresolved. 


Summary 

The  following  claims  for  the  efficacy  of  CAI  have  been  examined: 

1)  There  is  improved  retention  (achievement)  when  learning 
via  CAI . 

2)  Students  find  learning  via  CAI  to  be  enjoyable. 

3)  CAI  requires  less  time  for  the  student  to  reach  educa¬ 
tional  objectives. 

With  the  exception  of  time  savings,  the  findings  are  inconclusive 
but  seem  to  be  in  support  of  these  claims,  under  certain  conditions. 

For  example,  achievement  seems  to  improve  when  CAI  is  used  as  a  supple¬ 
ment  to  classroom  instruction.  Support  for  the  claim  of  time  savings 
for  the  student  is  virtually  unanimous . 

Historical  Perspective 

The  use  of  computers  in  education  began  about  two  decades  ago. 
Prior  to  1965,  use  of  computer  assisted  instruction  (CAI)  was  confined 
mostly  to  the  computer  industry  where  it  was  used  to  train  personnel 
(Suppes  $  Macken,  1978).  There  was  some  activity  in  elementary  school 
mathematics  before  this,  mostly  of  a  drill -and-practice  nature,  but  CAI 
did  not  really  reach  the  university  level  until  1967  when  a  first  year 
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course  in  Russian  was  offered  which  eliminated  all  classroom  work  (Suppes 
^  Mornings tar,  1969)  . 

Two  rather  lengthy  reports  have  been  made  of  computer  use  in 
California  [Comstock,  1969)  and  in  Massachusetts  (Rochart  §  Morton,  1975) 
which  demonstrate  use  in  higher  education  [in  institutions  beyond  high 
school)  .  Although  these  reports  included  both  courses  "with"  the  computer 
and  "about”  the  computer,  therewas  some  indication  of  substantial  use  of 
the  computer  in  teaching  subjects  other  than  computer  science.  At  that 
time,  the  largest  percentage  of  use  was  devoted  to  drill -and-practice  and 
problem  solving  rather  than  tutorial  mode . 

On  a  more  global  level,  Kearsley  [1976a)  has  examined  data  derived 
from  issues  of  Index  to  Computer  Based  Learning  [Lekan,  1970;  Hove  §  Wang, 
1973;  Wang,  1976)  and  attempted  to  show  trends  in  the  use  of  CAI  over 
time.  [These  indexes  have  some  shortcomings  as  a  data  source,  namely 
that  all  information  is  collected  via  questionnaire  which  is  voluntarily 
filled  out  and  returned  by  the  institution,  and  that  no  attempt  is  made 
to  verify  incoming  information.  The  fact  remains,  however,  that  the  Index 
is  likely  the  most  conplete  source  available.)  His  major  findings  are 
summarized  as  follows: 

1)  Instructional  strategy  -  Drill -and-practice  has  remained  the 
most  popular  over  the  period  with  tutorial  showing  a  sharp 
increase  in  use  in  1973  followed  by  a  decline  to  previous 
levels  by  1976. 

2)  Subject  matter  -  Mathematics  has  maintained  the  highest 
number  of  programs  over  the  entire  period  but  a  noticeable 
increase  in  courses  in  the  health  professions  occurred  be¬ 


tween  1973  and  1976. 
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-ource — pf  programs  -  Very  little  consistency  is  shown  in 
this  area  except  at  the  University  of  Illinois  and  at  the 
Coast  Community  College  which  have  consistently  produced 
the  largest  number  of  programs  since  1972.  (There  is  no  precise 
meaning  given  to  the  term  ’'program".  According  to  the  Index 
it  may  apply  to  the  name  of  a  course,  a  project,  or  a 
segment  of  a  course .  This  must  be  borne  in  mind  through¬ 
out  the  following  discussion  relating  to  numbers  and 
length  of  programs.) 

4)  Length  of  program  -  The  trend  has  been  toward  increasing 
length  of  programs.  When  average  student  completion  times 
are  compared  across  the  years,  one  sees  that  in  1970,  95% 
of  all  programs  were  completed  in  1-10  hours  with  48%  being 
completed  in  less  than  1  hour.  By  1976  only  88%  of  all 
programs  were  completed  in  1-10  hours  with  only  38%  being 
completed  in  less  than  1  hour. 

While  some  of  these  findings  may  sound  encouraging,  a  closer 
look  is  necessary.  Kearsley  (1976b)  has  made  an  in  depth  study  of  CAI 
use  in  1976  using  the  same  source  for  his  information  as  he  used  in  his 
comparison  of  1970-76.  Since  the  vast  majority  of  programs  continues  to 
be  less  than  10  hours  in  length,  it  appears  that  CAI  is  used  mainly  as 
an  adjunctive  method  of  teaching.  It  has  not  developed  into  a  viable 
replacement  for  classroom  instruction  as  was  hoped  in  the  early  days . 

Even  tutorial  programs  average  less  than  10  hours  in  length.  The  largest 
producer  of  programs,  Coast  Community  College,  with  519  programs  reported 
in  1976,  reports  total  average  completion  hours  to  be  324  or  less  than  2 
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hours  per  program. 

Another  interesting  finding  is  that  while  subject  matter  seems  to 
be  increasingly  varied,  only  191  of  the  programs  reported  in  the  1970 
Index  still  appear  in  the  1976  Index.  Although  some  of  this  may  be  due 
to  the  inaccuracy  of  the  data  collection  method,  at  least  part  of  the 
decrease  is  due  to  a  general  lack  of  long-term  funding  for  such  programs 
(Kearsley,  1976a) . 

As  one  peruses  the  literature  on  CAI,  several  facts  become  apparent. 
Firstly,  the  number  of  papers  reporting  evaluations  of  CAI  programs  has 
decreased  drastically  since  1974.  Secondly,  once  an  evaluation  report 
has  appeared,  nothing  further  is  published  in  the  more  common  educational 
technology  journals,  in  most  cases.  There  is  no  indication  as  to  the 
continuing  status  of  such  programs,  what  changes  have  been  made,  or 
the  results  of  such  changes.  This  leads  one  to  wonder  if  the  program 
was  developed  for  purposes  of  the  study  and  then  abandoned.  Lastly, 
there  seems  to  be  little  new  thought  on  the  subject  of  CAI.  To  be  sure, 
the  question  of  the  dehumanizing  effects  of  CAI  is  resurrected  periodic¬ 
ally,  and  a  few  authors  give  advice  on  setting  up  a  CAI  facility,  but  it 
is  only  by  examining  a  source  such  as  the  Index  to  Computer  Based  Learning 

that  one  has  much  idea  of  the  status  of  CAI  at  present.  This  review 
has  tried  to  demonstrate  the  inconclusiveness  of  earlier  findings  in 
evaluating  the  merits  of  CAI  and  the  need  for  more  evidence.  However, 
such  evidence  does  not  seem  to  be  readily  forthcoming. 

Once  again  we  must  turn  to  Kearsley  (1976c) .  He  outlines  a  number 
of  scenarios  as  possible  prospects  for  computer  based  education  (CBE)  in 
the  future.  In  the  first,  he  sees  CBE  as  an  enrichment  medium  much  like 
audio-visual  aides  currently  in  use.  The  data  on  program  length  support 
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this  scenario.  In  the  second  scenario,  CBE  is  the  major  mode  of  in¬ 
struction.  Little  support  exists  for  this.  The  third  scenario  describes 

/ 

CBE  as  a  research  tool  only,  and  the  fourth  calls  CBE  "an  educational 
dinosaur”  in  which  activity  levels  and  research  gradually  fade  into 
relative  non-existence.  The  increased  number  of  programs  listed  in  the 
1976  Index  (Wang,  1976)  seems  to  deny  the  last  scenario  but  the  1978 
Index  (Wang,  1978)  reports  a  decline  in  number  of  programs  for  the  major 
producers,  introducing  some  doubt  as  to  the  eventual  status  of  CAI . 

The  CAI  Facility  at  the  University  of  Alberta 

The  CAI  facility  at  the  University  of  Alberta  is  housed  in  the 
Division  of  Educational  Research  Services  (DERS)  in  the  Faculty  of 
Education.  The  hardware  consists  of  an  IBM  1500  system  with  19 
terminals.  Each  student  station  or  terminal  includes  a  CRT  with  a 
typewriter  keyboard,  a  light  pen,  an  image  projector,  and  an  audio 
play-record  unit  (Hunka,  1973).  The  facility  has  been  operational 
since  1968  with  the  amount  of  equipment  remaining  constant  since 
about  1970.  Hunka  (1973)  has  outlined  3  objectives  -  demonstration, 
research,  and  production  -  toward  which  energies  have  been  directed. 
Production  refers  to  the  use  of  CAI  courses  by  students.  Programs 
available  through  DERS  will  be  discussed  in  light  of  the  stated 
objectives . 

Demonstration 


Demonstrations  of  the  use  of  the  computer  have  been  made  to  many  groups 


. 
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both  on  and  off  campus,  including  thousands  of  school  children.  A 
program  titled  "DEMO”  gives  a  standard  introduction  to  the  terminal, 
lasting  5-10  minutes,  which  is  suitable  for  students  in  grade  6  or 
higher,  as  well  as  giving  the  student  access  to  a  wide  range  of  games 
drills,  tutorials,  and  simulations. 

"DEMO"  is  an  integration  under  one  course  name  of  many  sub-sections 
illustrating  the  CAI  capabilities  of  the  system. 

Research 


The  area  of  research  must  be  considered  to  include  not  only 
research  of  the  type  discussed  previously  (i.e.,  the  effects  of 
learning  via  CAI)  but  also  research  with  CAI.  Examples  of  the  latter 
include  such  things  as  administering  published  tests  through  the  CAI 
terminal  ("FIGUR"  -  Kagan’s  Matching  Familiar  Figures  test,  "tscs"  - 
Tennessee  Self  Concept  Scale) ,  experimenting  with  new  authoring 
languages  (Romaniuk,  1970,  using  VAULT),  and  using  the  terminal  to 
present  stimulus  items  in  studies  unrelated  to  CAI  directly  ("SEMSA"  - 
visual  research,  ’RECAL"  -  recall  of  nonsense  trigrams,  "PASSC"  -  paired 
associate  learning  task) .  Of  the  approximately  75  programs  available 
through  DERS,  almost  one  third  are  of  this  type,  developed  for  a  part¬ 
icular  research  project  but  not  subject  to  ongoing  usage. 

Production 


It  is  sometimes  difficult  to  distinguish  between  production  and 
research.  A  program  is  usually  produced  out  of  some  need,  either  to 
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offer  some  instruction  which  is  otherwise  unavailable  or  to  provide  an 
alternate  or  supplementary  means  of  instruction.  In  order  to  determine 
whether  or  not  the  program  is  fulfilling  that  need  adequately,  research 
is  undertaken. 

Most  of  the  programs  available  in  DERS  were  produced  (authored  and 
programmed)  there.  One  exception  is  "CARE1"  which  was  produced  at 
Pennsylvania  State  University  to  teach  teachers  early  identification  of 
handicapped  children.  It  has  been  made  available  through  DERS  to  teachers 
here . 

The  rest  of  the  programs  may  be  roughly  divided  into  two  classes: 
those  which  offer  drill-and-practice  or  simulations  as  a  means  of 
reinforcing  principles,  and  those  which  offer  all  or  part  of  the 
instruction  in  a  particular  course.  Many  of  the  drill-and-practice 
programs  are  for  elementary  school  mathematics  ("ACCUM"  -  arithmetic 
drill  for  grades  2  to  6) .  Simulations  are  available  for  junior  high 
school  students  ("FORST"  -  simulation  of  a  lumber  company  operation) , 
school  administrators  ("JARAC"  -  educational  decision  making) ,  and 
medical  students  (’’SLEEP”  -  simulation  of  anaesthesiological  procedures) , 
to  mention  a  few.  The  use  which  is  made  of  these  programs  is  variable. 

Programs  which  offer  instruction  as  all  or  part  of  a  course  are 
numerous  and  subject  to  more  regular  use  than  the  others.  With  few 
exceptions,  they  are  designed  for  students  at  post -secondary  in¬ 
stitutions  such  as  university  or  technical  institutions.  Table  1 
summarizes  the  major  programs. 

Examination  of  the  table  reveals  considerable  variability  in  the 
duration  of  courses.  (The  times  recorded  represent  average  student 
completion  times) .  The  shorter  ones  obviously  represent  only  one  unit 
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Table  1 

Summary  of  Major  Tutorial  Programs  at  University  of  Alberta 


Course  Name 

Content 

Duration 

User/ level 

AN0V1 

analysis  of  variance 

2-3  hrs 

university 

APLLC 

A  Programming  Lang,  learner 
control 

9  hrs 

gt.  10-univ.  - 

BASL1 

basic  electronics 

50  hrs 

univ./tech. 

CADAV 

health  hazard  appraisal 
program 

10-15  mins 

med.  §  nurs. 

CAI8 

intro,  to  color  T.V. 

20  mins 

gr.  10-tech. 

CARD1 

cardiology 

25  hrs 

2nd  yr.  med. 
students 

CARE1 

identification  handicapped 
chi ldren 

25  hrs 

university/ teachers 

CHNUR 

fundamentals  of  immunization 

lk  hrs 

comm,  health 
nurses 

COMP 

introduction  to  computers 

7k  hrs 

technical  school 

CWSH 

Coursewriter  II 

15  hrs 

gr.  10-univ. 

FRAND 

French 

25  hrs 

jr.  §  sr. 
high  school 

FUNDP 

fundamentals  of  data 
processing 

22  hrs 

gr. 7- adult 

KAMRA 

intro  to  photography 

45  mins 

univ./tech 

PHAPM 

intro,  to  autonomic 
nervous  system 

2  hrs 

univ. 

PHONI 

intro,  to  teaching 
phonics 

15  hrs 

univ. 

STAT1 

statistics  §  research 

70  hrs 

univ. 

TESL 

Eng.  as  a  2nd  lang. 

lk  hrs 

gr.  4-univ. 

TOOLS 

tools  in  metal  work 

6k  hrs 

univ. 

. 
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of  a  regularly  taught  course  while  the  longer  ones  my  represent  the 

total  instruction  in  a  course.  From  this  list,  the  program  which  is  of 

/ 

interest,  and  which  will  be  described  in  detail,  is  "ST ATI". 

Description  of  STAT1 

The  information  in  this  segment  comes  from  Hunka,  Romaniuk  and 
Maguire  (1976)  as  well  as  the  author's  personal  experience  with  the 
program. 

STAT1  is  a  computer-assisted  instruction  program  in  statistics 
offered  at  the  graduate  level  as  part  of  the  course  title,  Educational 
Psychology  502.  It  includes  topics  ranging  from  descriptive  statistics 
to  analysis  of  variance  and  research  design  (See  Table  2) . 

The  program  uses  the  tutorial  mode  of  instruction  which  is 
supplemented  with  problem-solving  and  a  few  drill -and-practice  exercises 
incorporated  into  the  program  where  appropriate.  Reviews  of  concepts 
are  offered,  with  the  student  making  the  choice  to  review  or  advance 
to  new  material.  Students  may  also  review  segments  at  any  time  by 
selecting  the  correct  label  and  moving  to  that  portion  of  the  course. 

Student  responses  are  entered  at  the  terminal  either  by  using  a 
lightpen  to  point  at  a  response  on  the  screen  or  by  entering  a  typed 
response  at  the  keyboard.  Keyboard  responses  my  involve  entering  the 
number  or  letter  of  the  answer  choice  (as  in  multiple  choice  questions) 
or  constructing  a  free  response  (either  words  or  numbers)  as  the  situation 
demands.  Feedback  on  correct  responses  often  takes  the  form  of  encourag¬ 
ing  or  congratulatory  messages.  Incorrect  responses  receive  feedback 
in  the  form  of  hints  and  "try  again"  messages.  The  correct  answer  is 
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Table  2 

Course  Content  of  the  Computer -As sis ted  Program  Called  STAT1 


STAT1 

Segment# 

0 

3 

20 

4 

5 

7 

8 
9 

10 

6 

11 

12 

13 

14 
16 
17 
19 
25 


Description 
Introduction 
Descriptive  Statistics 

Area  under  the  Normal  Curve  of  the  Binomial  Distribution 

Prediction  and  Correlation 

Multiple  and  Partial  Correlation 

Knowledge  of  Terms  in  Inferential  Statistics 

Random  Sampling  Distribution  of  the  Mean 

Properties  of  Samples  drawn  from  the  Random  Sampling 
Distribution 

Hypothesis  Testing 

t- tests  and  Confidence  Intervals 

Purpose  of  One-Way  Analysis  of  Variance  (ANOVA) 

Computation  of  One-Way  ANOVA 

Intuitive  Approach  to  the  Rationale  of  One-Way  ANOVA 
Algebraic  Approach  to  the  Rationale  of  One-Way  ANOVA 
Scheffe  and  Tukey  Tests 
Interaction  and  Two-Way  ANOVA 
Chi  Square 

Using  a  Computer  to  Process  Data 
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usually  only  given  after  several  incorrect  attempts . 

At  the  commencement  of  the  program  students  are  given  a  paper- and- 
pencil  arithmetic  test  to  determine  the  level  of  competency  in  basic 
skills.  All  other  testing  associated  with  STAT1  is  achievement  testing. 
Ten  unit  exams  are  given,  six  of  which  are  taken  at  the  terminal. 

"Right"  or  "wrong”  is  the  only  feedback  offered  at  the  time.  A 
summary  of  performance  is  given  at  the  end  of  each  exam  accompanied 
by  a  suggestion  for  review  if  achievement  falls  below  a  certain  level 
(usually  about  75%) . 

The  average  student  completion  time  for  the  entire  program  is 
about  70  hours,  excluding  time  spent  on  lab.  assignments.  At  the  end 
of  each  unit  but  before  the  exam,  an  attitude  questionnaire  is  auto¬ 
matically  presented  to  the  student  at  the  terminal  (See  Appendix  A) . 

The  attitude  questions  are  answered  on  a  7  point  Likert  scale  with 
ratings  on  an  agree/ disagree  format.  The  questionnaire  attempts  to 
determine  such  factors  as  whether  the  student  found  the  material 
interesting,  if  he  was  already  familiar  with  the  material,  and  whether 
he  would  have  preferred  classroom  instruction. 

The  program  was  first  offered  in  its  entirety  in  the  summer  of 
1975.  It  was  revised  on  the  basis  of  data  collected  during  the  summer, 
and  has  been  offered  every  summer  since  then.  The  STAT1  program  was  also 
utilized  following  the  same  format  in  the  winter  of  1976-77.  Since 
then,  Winter  Session  students  have  used  STAT1  along  with  regular  class¬ 
room  instruction.  The  main  difference  between  STAT1  taken  in  Summer 
Session  vs.  Winter  Session  is  time.  Summer  students  complete  the 
program  over  a  6 -week  period  while  winter  students  have  about  8  months 


, 
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in  which  to  complete  the  program.  The  implications  of  the  shorter  time 

period  in  which  to  complete  the  program  are  a  more  concentrated  approach 

/ 

(in  terms  of  daily  attendance  at  the  terminal)  by  the  students,  and  lack 
of  interference  from  other  courses.  The  exact  effects  of  the  varying 
times  for  completion  are  unknown  but  each  may  have  positive  and  negative 
aspects.  For  example,  over  an  8  month  period,  students  could  pace  them¬ 
selves  at  a  rate  of  3  hours  per  week  rather  than  3  hours  per  day, 
allowing  more  time  for  integration  of  new  material,  and  more  review 
time.  At  the  same  time,  the  tendency  may  be  to  leave  the  program 
when  work  from  other  courses  piles  up,  and  then  try  to  catch  up  by 
working  intensively  at  STAT1  for  several  days  in  a  row.  Generally 
it  would  seem  that  6-week  courses  give  the  students  fewer  options  for 
budgeting  their  time  but  less  interference  from  the  demands  of  other 
courses . 

Students  are  generally  from  the  various  education  departments  (Ed¬ 
ucational  Psychology,  Educational  Foundations,  Elementary  Education,  etc.) 
but  students  from  other  faculties  also  take  the  course  (e.g.,  it  is 
compulsory  for  students  enrolled  in  the  Master's  Degree  program  in  Physical 
Education) . 

Besides  test  scores  and  attitude  ratings,  many  types  of  data  are 
collected  on  the  students.  These  include  answer  analysis  information 
(response  time,  correct,  wrong,  and  unrecognized  responses) ,  time  taken 
to  complete  each  segment,  previous  knowledge  of  statistics,  and  comments 
made  by  students  and  typed  in  at  the  terminal. 

STAT1  is  the  most  lengthy  program  available  through  DERS.  It 
covers  the  topic  areas  in  a  detailed  and  organized  fashion  which  should 
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allow  the  student  to  carry  through  without  much  assistance  from  an 

instructor  except  for  individual  problems .  The  program  is  always  accom- 

/ 

panied  by  lectures  on  topics  related  to  research  design,  such  as  re¬ 
liability,  validity  and  how  these  factors  are  affected  by  the  design  of 
an  experiment.  These  lectures,  however,  constitute  about  16  hours  in 
the  total  program  compared  to  the  average  terminal  time  of  about  70 
hours . 

It  is  evident  from  reviewing  the  literature  that  little  follow-up 
research  on  the  effectiveness  of  CAI  programs  is  published.  As  an 
attempt  to  fill  the  need  for  follow-up  a  study  is  proposed  to  investigate 
the  retrospective  perceptions  of  students  who  have  completed  the  STAT1 
program,  and  also  to  gain  some  measure  of  follow-up  achievement  in  the 
program. 

The  specific  questions  to  be  addressed  by  the  present  study  are: 

1)  Did  achievement  scores  six  months  to  2  years  after  course  completion 
differ  from  those  gained  at  the  time  of  initial  testing? 

2)  Did  retrospective  perceptions  of  the  STAT1  program  differ  from  those 
noted  at  the  time  of  initial  testing? 

3)  Did  the  amount  of  reported  use  of  statistics  have  any  relation¬ 
ship  to  follow-up  perceptions  or  achievement  scores? 

4)  What  were  the  specific  factors  which  tended  to  be  associated  with 
positive  and  negative  attitudes  toward  STAT1? 

How  helpful  were  features  outside  STAT1 ,  such  as  laboratory  assign¬ 
ments  and  research  design  classes,  perceived  to  be? 


5) 
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Chapter  3 
Methodology 


Subjects 

The  subjects  employed  in  this  study  included  all  those  for  whom 
an  address  could  be  obtained  who  had  conpleted  the  STAT1  program  as 
it  was  used  in  Educational  Psychology  502  during  Winter  Session  1976-77, 
Summer  Session  1977,  Spring  Session  1978,  and  Summer  Session  1978.  The 
total  number  of  students  completing  the  program  in  the  sessions  listed 
was  114.  Of  these,  addresses  were  obtained  for  a  total  of  93  persons. 

The  remaining  23  subjects  were  spread  fairly  evenly  over  all  groups, 

(See  Table  3),  with  the  1977  group  losing  the  greatest  number  of 
potential  subjects.  The  groups  are  referred  to  as  Group  76,  Group  77, 
and  Group  78,  the  number  denoting  the  year  in  which  the  program  was  begun. 

Although  STAT1  has  been  in  full  operation  since  Summer  Session  1975, 
persons  completing  the  program  before  Winter  Session  1976-77  were 
eliminated,  for  two  reasons.  Firstly,  it  was  felt  that  these  people 
would  be  very  difficult  to  contact,  given  the  length  of  time  intervening. 
Secondly,  the  course  instructors  believed  that  the  early  groups  of  students 
had,  in  fact,  been  treated  differently.  Owing  to  the  novelty  of  the 
program,  and  resultant  enthusiasm,  the  instructors  felt  that  they  had 
spent  more  time  with  the  students,  assisting  and  encouraging  them.  There 
had  also  been  several  changes  made  to  the  program,  based  on  the  experience 
with,  and  feedback  from,  the  early  groups  of  students.  The  STAT1  program 
has,  however,  remained  fairly  stable  since  the  commencement  of  Winter 
Session  1976-77. 

The  groups  chosen  for  this  study  were  assumed  to  be  equal  except 
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Table  3 

Examinee  Sampling  and  Response  Rates 


No.  completing  No.  for  whom  Questionnaires 

Course  Addresses  Returned  %  Returned 

Available 


Group 


76 

33 

27 

10 

37 

77 

35 

21 

10 

48 

78 

49 

45 

24 

53 
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on  one  factor:  Group  76  (G76) ,  completed  the  program  over  26  weeks 
while  Groups  77  (G77)  and  78  (G78)  completed  the  program  over  six 
weeks  of  intensive  study.  No  information  was  available  on  persons  who 
dropped  the  course  before  completion.  In  1976-77  five  persons  withdrew 
from  EdPsy  502,  but  two  reregistered  in  1977  and  completed  the  course. 
There  were  three  dropouts  in  Summer  1977  and  three  in  Summer  1978. 
Materials 

Since  the  main  focus  of  this  study  was  on  retrospective  perceptions 
of  the  STAT1  program  and  on  retention  over  time,  it  was  necessary  to 
develop  paper- and -pencil  questionnaires  which  could  provide  measures  of 
perception  and  achievement  for  comparison  with  the  initial  measures  of 
perception  and  achievement  taken  at  the  completion  of  the  program. 

STAT1  Opinionaire 

This  questionnaire  (See  Appendix  "B")  was  constructed  by  modification 
and  expansion  of  the  questionnaire  which  students  completed  at  the  end 
of  STAT1.  Items  which  were  considered  irrelevant  to  the  study  (e.g.  those 
requesting  demographic  data)  were  eliminated,  while  others  were  reworded 
to  better  fit  the  post  hoc  nature  of  the  study.  In  addition,  all  three 
instructors  involved  in  the  program  were  asked  to  submit  suggestions  for 
items  which  would  tap  particular  areas  of  concern,  especially  content  and 
emphasis  of  the  program. 

The  result  was  a  31  item  questionnaire,  the  majority  of  which  could 
be  responded  to  by  marking  the  appropriate  area  on  a  seven  point  Likert 
scale.  Estimated  time  for  completion  of  the  total  questionnaire  was  three 


to  five  minutes . 
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Achievement  Questionnaire 

The  greatest  difficulty  in  gaining  a  reliable  estimate  of  achieve- 

/ 

ment  was  seen  to  be  the  time  required  of  the  subject  to  complete  a 
questionnaire  which  adequately  sampled  the  course  material.  A  decision 
was  made,  based  on  the  writings  of  Shoemaker  (1970a,  1970b,  1971)  and 
Lord  (1962) ,  to  use  matrix  sampling  procedures  to  estimate  mean  achieve¬ 
ment  scores,  and  thus  reduce  greatly  the  time  commitment  required  of  an 
individual  subject. 

Briefly,  Shoemaker  (1970a)  has  shown  that  if  test  items  are  divided 
into  "t"  subtests  and  each  subtest  is  administered  to  a  different  set  of 
examinees ,  the  results  obtained  on  each  subtest  may  be  averaged  to  provide 
a  single  estimate  of  the  population  mean  and  variance.  Shoemaker  (1970b) 
has  further  demonstrated  that  the  degree  of  accuracy  in  estimating  the 
population  mean  and  variance  using  item-examinee  sampling  procedures  is 
a  function  of  the  number  of  observations  obtained,  rather  than  the  pro¬ 
cedure  itself.  An  observation  is  defined  as  a  single  item  on  a  single 
subtest  administered  to  a  single  examinee.  The  total  nmrber  of  observa¬ 
tions  is  then  determined  by  Number  of  subtests  x  Number  of  items  per  sub¬ 
test  x  Number  of  examinees  per  subtest.  In  general,  ’’the  greater  the 
number  of  observations  used,  the  smaller  the  standard  error  of  estimating 
u  and  <72.”  (Shoemaker,  1971,  p.  218).  Beyond  1200  observations,  gains 
in  accuracy  of  estimation  are  small,  provided  adequate  item  sampling  has 
occurred. 

A  bank  of  132  items  was  available  from  those  items  which  appeared 
on  the  final  examinations  used  for  each  of  the  three  groups.  These  items 
were  grouped  according  to  subject  matter,  following  the  course  outline 
for  STAT1  (See  Table  2) .  The  resulting  10  groups  of  subject  matter 
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were  then  compressed  into  five  larger  groups  by  combining  subject  areas 
which  seem  to  go  together  (e.g.  Descriptive  Statistics  and  Area  Under 
the  Normal  Curve  were  combined) . 

Six  15-item  stratified  achievement  tests  were  then  formed  by  random 
sampling  without  replacement  from  each  of  the  five  groups  of  items,  using 
a  table  of  random  numbers.  Subject  matter  grouping  and  item  selection 
occurred  as  shown  in  Table  4. 

Three  of  the  subtests  were  mailed  to  15  examinees  each  while  the 
remaining  three  subtests  were  mailed  to  16  examinees  each,  for  a  potential 
total  of  (3  x  15  x  15)  +  (3  x  15  x  16)  =  1395  observations. 

Procedure 

A  package  was  prepared  for  mailing  to  each  of  the  93  subjects.  Each 
package  contained  a  copy  of  the  STAT1  opinionaire,  a  15- item  achievement 
test,  a  stamped  return  envelope  addressed  to  the  author,  and  a  covering 
letter  (See  Appendix  "C") .  Subjects  living  in  Edmonton  were  contacted 
by  telephone  to  explain  the  nature  of  the  study  and  to  elicit  their  co¬ 
operation.  Questionnaires  were  mailed  on  January  4,  1979  with  return 
requested  by  January  22,  1979.  Subjects  living  in  Edmonton  were  contacted 
again  by  telephone  if  their  completed  questionnaires  were  not  received 
by  January  24. 

All  questionnaires  were  coded  as  subjects  were  not  required  to  identify 
themselves.  Care  was  taken  to  ensure  that  approximately  equal  numbers  from 
each  group  received  each  of  the  six  achievement  tests.  The  time  estimated 
to  complete  both  questionnaires  was  about  45  to  50  minutes,  the  bulk  of 
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Table  4 

Sub test  Item  Distribution  According  to  Content 


Subject  matter 

Total  items 

No.  selected  for 
each  subtest 

Descriptive  statistics, 
area  under  normal  curve 

33 

4 

Prediction  and  correlation, 
multiple  and  partial  correl. 

25 

3 

Hypothesis  testing, 
t-test,  Chi  square 

25 

3 

One  way  ANOVA,  interaction, 
Tukey  and  Scheffe  tests 

20 

7 

Research  design,  computer 
processing  of  data 

29 

3 

Total 

132 

15 

. 
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this  time  being  devoted  to  the  achievement  test.  Subjects  were  instructed 
in  the  covering  letter  to  treat  the  achievement  test  as  an  open  book  exam 
since  this  was  the  format  used  by  the  course  instructors.  Response  rates 
are  shown  in  Table  3. 

The  overall  response  rate  was  44  persons  or  47%  of  the  total  possible. 
Of  the  44  subjects  who  responded  to  the  questionnaires,  39  (42%  of  total 
possible)  completed  both  the  achievement  test  and  the  opinionaire.  The 
remaining  five  subjects  completed  only  the  opinionaire.  The  initial 
response  from  subjects  contacted  by  telephone  resulted  in  52  persons 
agreeing  to  participate  in  the  study.  When  the  second  telephone  contact 
was  made  following  the  deadline  for  returning  the  questionnaires,  a  brief 
interview  was  conducted  to  determine  the  precise  reason  for  the  delay. 

For  example,  subjects  were  reminded  of  the  deadline  and  asked  if  they 
had  forgotten  or  had  changed  their  minds  about  participating.  If  the 
latter  response  was  given,  the  matter  was  pursued  further.  Where  subjects 
stated  that  it  would  take  longer  than  45  minutes  to  complete  the  achieve¬ 
ment  questionnaire,  they  were  asked  to  complete  the  opinionaire.  Five 
subjects  agreed  to  do  this. 

Analysis  of  the  Data 

Achievement 

The  achievement  test  scores  were  converted  to  percentages  and  the 
mean  and  standard  deviation  were  computed  for  39  subjects .  The  final 
course  grades  received  by  these  subjects  were  gathered  and  the  mean  and 
standard  deviation  also  conputed.  A  Pearson  correlation  coefficient  was 
calculated  between  the  two  sets  of  achievement  scores . 
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Opinion  Data 

Each  of  the  27  opinionaire  items  which  required  the  subject  to 
rate  their  preference  on  a  seven-point  scale  was  examined.  The  integer 
value  of  the  rating  level  selected  by  the  individual  was  used  as  the 
score  for  that  person  on  that  item,  with  the  response  indicating  least 
agreement  (Strongly  Disagree)  or  most  negative  attitude  being  scored 
one  and  the  response  indicating  the  most  agreement  (Strongly  Agree)  or 
the  most  positive  attitude  being  scored  seven.  The  scores  for  the 
following  items  were  reflected  to  conform  to  this  scoring  regime: 

Item  7  It  would  have  been  easier  to  learn  the  STAT1  material 
by  classroom  instruction  rather  than  by  computer 
instruction. 

Item  8  The  material  in  the  program  was  boring  --  interesting. 

Item  11  What  portion  of  the  material  covered  in  STAT1  had  you 
previously  been  taught?  None  --  All. 

The  mean  and  standard  deviation  of  the  scores  on  each  item  were 
calculated  and  Pearson  correlation  coefficients  were  computed  for  all 
possible  pairs  of  items  as  well  as  between  opinionaire  items  and  achieve¬ 
ment  test  scores  and  final  grades .  Only  correlations  between  scores 
related  to  achievement,  expressed  liking  for  STAT1  and  perceived  use¬ 
fulness  of  the  material  learned  will  be  discussed  in  the  next  chapter. 
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Giapter  4 

Results  and  Discussion 


The  data  collected  for  analysis  consisted  of  responses  to  27 
opinionaire  items  plus  achievement  test  scores,  final  grade  scores, 
and  end-of-course  opinion  data.  In  all,  44  subjects  responded  to 
the  follow-up  questionnaires;  39  of  these  responded  to  both  question¬ 
naires.  The  remaining  five  subjects  declined  to  complete  the  achieve¬ 
ment  test.  At  initial  contact,  52  persons  verbally  agreed  to  partici¬ 
pate  in  the  study.  However,  of  these  52,  only  38  partially  or  com¬ 
pletely  fulfilled  the  commitment.  Six  persons  who  had  been  contacted 
by  mail  only  also  participated.  The  overall  response  was  dis¬ 
appointing  in  light  of  the  initial  cooperation  received.  Therefore, 
an  attempt  was  made  to  ascertain  the  reasons  for  subjects’  failure 
to  cooperate.  The  subjective,  informally  gathered  feedback  will  be 
integrated,  where  possible,  into  the  discussion  of  the  findings. 

The  possibility  of  having  a  biased  sample  was  considered  and  a 
number  of  checks  were  made.  First,  the  final  course  grades  for 
respondents  and  non- respondents  were  rank  ordered  to  determine  if 
respondents  tended  to  cluster  at  one  end  or  the  other.  Respondents  were 
evenly  distributed  throughout  the  range.  Next  the  proportion  of 
respondents  from  each  of  the  university  departments  represented  were 
compared.  Educational  Psychology  students  had  the  greatest  response 
rate  (601)  with  other  Education  departments  having  response  rates  25% 
or  higher.  Only  Physical  Education  was  poorly  represented  with  a  15% 
response  rate.  The  high* Educational  Psychology  response  rate  is 
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explained  by  the  fact  that  two- thirds  of  all  students  in  the  1978 
Group  came  from  this  department  so  they  were  the  most  accessible  to 
this  study.  The  low  response  rate  from  Physical  Education  students  is 
explained  by  the  fact  that  two-thirds  of  the  students  in  this  depart¬ 
ment  were  in  the  1976-77  Group  and  only  two  were  still  in  the  city. 
Finally,  the  Educational  Psychology  students  were  classified  into  the 
three  areas  of  study  within  the  department  (Basic,  Special  Education, 
and  Counselling)  to  see  if  one  area  was  proportionally  better  rep¬ 
resented  than  the  others.  Counselling  students  had  a  much  higher  rep¬ 
resentation  (85%  compared  to  50%)  than  the  other  two  areas.  Thus  it 
may  be  said  the  respondent  group  is  more  representative  of  counselling 
students  in  Educational  Psychology  than  of  any  other  group  participating 
in  the  study. 

Achievement  Data 

Scores  were  examined  in  an  overall  manner  as  well  as  according  to 
groups,  as  shown  in  Table  5.  For  the  39  subjects  who  wrote  the  Achieve¬ 
ment  Questionnaire,  the  mean  score  was  57.79%  with  a  standard  deviation 
of  18.27%.  The  mean  of  the  final  course  grades  for  these  subjects 
was  79.45%  with  a  standard  deviation  of  10.74%  (n=38,  see  note,  Table  5). 
The  correlation  between  final  course  grades  and  achievement  test  score 
was  .45  (p  .01). 

For  all  groups  tested,  mean  achievement  scores  dropped  to  more  than 
15%  below  mean  final  grades  while  the  standard  deviation  doubled  or 
tripled  that  of  the  final  grades .  No  statistical  tests  were  performed 
because  a)  G76  and  G77  contained  very  small  numbers  of  subjects  for 
whom  complete  data  were  available,  and  b)  multiple  matrix  sampling 


Table  5 


Mean  Achievement  Test  Scores  and  Final  Grades  By  Group 


26  weeks 

6  weeks 

6  weeks 

1976-77 

1977 

1978 

Achievement  Scores 

51.43* 

67.71 

55.83 

(n=38) 

(21.45) 

(15.16) 

(17.43) 

-3 

n=7 

n=7 

n=24 

Final  Grades 

73.86  , 

85.14 

79.42 

(n=38) 

(19. 87)1 2 3 4 

(3.67) 

(7.72) 

n=7 

n=7 

n=24 

Total  Class  Final 

76.04 

81.74 

76.43 

Grades 

(6.98) 

(5.97) 

(9.43) 

(n=114) 

n*32 

n=32 

n=50 

1  all  values  are  expressed  in  % 

2  standard  deviations  of  percents  in  parentheses 

3  n=8  subjects  completed  the  Achievement  Questionnaire  but  one  was 
eliminated  from  this  calculation  as  she  had  audited  Educational 
Psychology  502  and  no  final  grade  was  available. 

4  this  discrepant  value  occurs  because  one  subject  in  the  sample 
did  not  finish  the  course  and  received  a  failing  grade  of  31.32%. 
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procedures  were  used  to  gather  the  achievement  test  data,  thus  raising 
doubts  about  the  validity  of  assuming  these  small  amounts  of  data  to 
be  truly  representative  of  achievement  for  the  entire  group.  In  order 
to  estimate  population  means  and  variances  using  matrix  sampling  pro¬ 
cedures,  Shoemaker  (1970b)  has  demonstrated  that  standard  errors  of 
estimate  are  small  when  more  than  1200  observations  are  gathered.  In 
this  case,  less  than  600  observations  were  obtained  overall,  and  the 
number  obtained  for  each  group  is  therefore  too  small  for  reliable 
estimation  of  population  parameters.  In  Group  76  (G76)  and  Group  77 
(G77) ,  some  subtests  were  responded  to  by  only  one  person,  hence  the 
variance  of  the  items  is  zero  and  standard  errors  of  estimate  cannot 
be  calculated. 

The  difference  between  achievement  test  scores  and  final  course 
grades  for  study  participants  was  very  similar  for  all  groups  despite 
the  fact  that  the  time  elapsed  since  taking  the  course  ranged  from  six 
months  to  two  years.  It  might  be  hypothesized  that  persons  taking  the 
course  in  1976-77  would  have  made  more  use  of  the  material  learned,  and 
hence  maintained  a  higher  level  of  achievement.  This  did  not  appear 
to  be  the  case,  however.  Reported  use  of  statistical  knowledge  for 
thesis  and  general  work  was  about  the  same  for  all  groups,  ranging  from 
a  mean  of  3.57  on  a  seven-point  scale  to  4.71  for  thesis  use,  and  from 
3.44  to  4.33  for  general  use.  There  was  no  evidence  to  suggest  that  only 
the  highest  achievers  in  the  1976-77  group  participated  since  the  rank 
order  on  final  grades  was  1,  3,  8,  11,  18,  20,  21  and  34  out  of  35  for 
those  who  participated  in  this  study. 

An  alternative  explanation  is  that  taking  the  program  over  a  longer 
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period  of  time  (over  an  eight  month  session  as  opposed  to  six  weeks) 
allowed  for  better  integration  of  the  subject  matter  resulting  in  the 
maintenance  of  higher  achievement  levels.  This  hypothesis  was  partially 
supported  by  reports  from  students  taking  the  program  over  a  six  week 
period  who  stated  that,  "There  is  too  much  material  to  be  covered  in 
such  a  short  time".  The  motivation  for  many  students  to  take  the  program 
over  the  shorter  sessions  was  reported  by  them  as  being  "to  get  it  over 
with  as  fast  as  possible”,  and  few  acknowledged  the  value  of  the  know¬ 
ledge  gained  even  in  helping  them  to  understand  and  critique  published 
articles . 

Usefulness  and  Expressed  Liking  for  STAT1 

In  order  to  gain  some  understanding  of  the  students'  perceptions  of 
STAT1  regarding  learning  the  material  via  CAI,  attention  was  paid 
particularly  to  those  items  on  the  Opinionaire  which  dealt  with  reported 
usefulness  of  the  program  and  expressed  liking  for  it. 

The  questions  which  related  to  liking  STAT1  were  as  follows: 

V8  It  would  have  been  easier  to  learn  the  STAT1  material  by  class¬ 
room  instruction  than  by  computer  instruction. 

V9  The  material  in  the  program  was  boring  --  interesting. 

VI 8  I  liked  taking  statistics  via  computer- assisted  instruction. 

V19  Taking  statistics  via  computer- ass is ted  instruction  was  a  good 
investment  of  my  time. 

V20  I  would  recommend  the  STAT1  program  to  other  graduate  students  or 
educational  personnel. 

The  intercorrelations  of  these  variables  are  shown  in  the  Table  6. 
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Table  6 


Intercorrelations  of  Achievement  Variables  and  Variables 

Related  to  Liking  ST ATI 
(n=44) 


Variable 

V8 

V9 

VI 8 

VI 9 

V20  ACHIEVE 

FINAL 

V8 

1.00 

.  25* 

47*** 

.25* 

.56*** 

.39** 

.007 

V9 

1.00 

>56*** 

.46*** 

59*** 

.42** 

.16 

VI 8 

1.00 

74*** 

.58*** 

.41** 

oo 

o 

• 

VI 9 

1.00 

.70*** 

.44** 

.41** 

V20 

1.00 

.38** 

.24 

ACHIEVE 

1.00 

.45** 

FINAL 

1.00 

*  significant  at  .05 

**  significant  at  .01 

***  significant  at  .001 
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The  correlations  of  variable  8  (V8)  with  variables  9  (V9)  and  19 
(V19)  represent  the  minimum  values  which  are  significant  at <*=.05, 
suggesting  that  preference  for  classroom  learning  over  CAI  is  weakly 
related  to  how  interesting  the  material  is  perceived  to  be  and  to 
whether  taking  statistics  via  CAI  was  a  good  time  investment.  Perhaps 
students  were  reacting  to  the  freedom  one  has  to  take  a  break  from  the 
computer  at  any  time  and  thus  alleviate  boredom,  fatigue,  or  frustra¬ 
tion  at  one’s  own  choosing. 

Other  variables  are  moderately  to  highly  (V19  with  VI 8  and  V20) 
intercorrelated,  suggesting  that  responses  of  students  to  one  aspect 
of  liking  STAT1  tend  to  be  quite  similar  to  their  responses  to  other 
aspects  of  liking  STAT1.  Table  7  illustrates  the  mean  rating  on  items 
which  expressed  liking  for  STAT1,  and,  where  possible,  compares  the  average 
response  levels  on  follow-up  with  those  obtained  at  the  completion  of  the 
course.  No  such  comparison  is  possible  for  Group  76  since  this  opinionaire 
was  developed  in  1977.  The  original  response  levels  are  for  the  entire 
class  (for  G77,  n=32;  for  G78,  n=36) .  Since  the  opinionaires  were  done 
anonymously,  it  was  not  possible  to  separate  out  only  those  who  partici¬ 
pated  at  follow-up. 

From  Table  7  it  can  be  seen  that  the  follow-up  scores  among  the 
groups  are  markedly  similar  and  that  in  no  case  do  the  follow-up  scores 
differ  from  the  original  scores  for  the  group  by  more  than  .6.  Despite 
a  considerable  drop  in  mean  achievement  scores,  the  students  tended  to 
express  continued  positive  attitudes  toward  the  STAT1  program. 

At  this  point,  it  might  be  useful  to  reiterate  that  the  subject 
being  considered  here  is  the  CAI  program  and  students'  perceptions  of 
it  as  a  learning  device.  It  may  not  be  inferred  from  these  data  that 
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Table  7 


Original  and  Follow-up  Means 
of  Expressed  Liking  for  STAT1 


Variable 

G76 

Follow-up 

G77 

Original 

Follow-up 

G78 

Original 

Follow-up 

V8 

5.7 

5.3 

5.8 

5.6 

5.4 

V9 

4.8 

5.2 

4.9 

4.9 

4.5 

VI 8 

6.1 

5.9 

5.9 

5.7 

5.6 

VI 9 

5.1 

5.8 

5.8 

5.5 

5.3 

V20 

4.9 

5.8 

5.2 

5.4 

5.2 
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students  are  expressing  their  feelings  toward  the  subject  matter 
per  se.  As  will  be  shown  later,  the  students  reported  only  moderate 
use  made  of  the  material  learned.  In  general,  informal  interviews 
conducted  with  participants  and  non-participants  revealed  some  very 
negative  attitudes  toward  the  content  or  general  area  of  the  program, 
i.e.  statistics.  Comments  included  suggestions  that  a)  statistics 
had  no  applicability  for  students  in  counselling  psychology,  b)  the  only 
content  of  value  was  that  covering  research  design,  and  c)  statistics 
could  be  used  to  manipulate  data  to  give  whatever  results  are  desired, 

”so  why  bother?”.  Not  all  students  displayed  such  negativism  but  the 
numbers  who  did  were  sufficient  to  warrant  mention. 

Opinionaire  items  which  related  to  the  perceived  usefulness  of 
STAT1  are  as  follows: 

V7  I  found  the  STAT1  program  was  useful  in  helping  me  learn  statistics. 

VI 9  Taking  statistics  via  CAI  was  a  good  investment  of  my  time. 

V22  How  useful  was  (is)  the  material  learned  in  Ed.  Psych.  502  in 
doing  your  thesis? 

V25  How  useful  was  (is)  the  material  learned  in  Ed.  Psych.  502  in  your 
general  work  situation. 

The  intercorrelations  of  these  variables  are  shown  in  Table  8. 

One  opinionaire  statement  appears  in  both  categories  of  liking 
and  usefulness  since  it  was  felt  that  a  judgement  as  to  whether  taking 
STAT1  was  a  good  investment  of  a  student’s  time  could  easily  be  related 
to  both. 

The  intercorrelations  of  the  ’’usefulness”  variables  are  less  clear- 
cut  than  for  the  "liking”  variables.  It  appears  to  be  the  case  that 
use  of  statistical  knowledge  in  the  general  work  situation  is  unrelated 
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Table  8 

Intercorrelations  of  Achievement  Variables  and  Variables 
Related  to  the  Usefulness  of  STAT1 


Variable 

V7 

VI 9 

V22 

V25 

ACHIEVE 

FINAL 

V7 

1.00 

.45** 

.16 

.36** 

.30* 

VI 9 

1.00 

.68*** 

.38** 

.44** 

.41** 

V22 

1.00 

.25 

.31 

.44** 

V25 

1.00 

.26 

.08 

ACHIEVE 

1.00 

.45** 

FINAL 

1.00 

*  significant  at  .05 

**  significant  at  .01 

***  significant  at  .001 


47 


to  any  of  the  other  variables  except  V19  (Taking  statistics  by  computer- 
assisted  instruction  was  a  good  investment  of  my  time) .  Usefulness  of 
statistical  knowledge  in  doing  a  thesis,  however,  is  moderately  related 
to  "I  found  the  STAT1  program  was  useful  in  helping  me  learn  statistics.” 
(r=.45,  p< . 01)  and  more  strongly  related  to  ’’Taking  statistics  via 
computer- ass is ted  instruction  was  a  good  investment  of  my  time.”  (r=.68, 
pc.001).  Quite  obviously,  those  students  who  found  the  material  useful 
in  doing  a  thesis  might  be  expected  to  report  the  program  as  having  been 
a  good  investment  of  their  time.  The  questions  which  arise  from  this 
are  the  amount  of  actual  use  reported  by  the  students,  and  perceived 
value  of  the  time  invested  in  the  program  at  follow-up  compared  to  when 
the  program  was  taken. 

Table  9  illustrates  the  mean  ratings  given  by  the  subjects  in  the 
study  sample  as  well  as  the  class  mean  ratings  taken  at  the  time  of 
finishing  the  program,  where  these  are  known. 

Table  9  shows  lower  mean  ratings  for  all  groups  on  the  variables 
dealing  with  application  of  statistical  knowledge  to  thesis  or  general 
work  than  for  variables  dealing  with  the  usefulness  of  STAT1  in  learning 
statistics.  Generally  subjects  expressed  the  sentiment  that  although 
they  had  liked  taking  the  course  by  computer,  and  had  achieved  well  on 
examinations,  they  felt  they  had  attained  little  true  understanding  of 
the  material  and  hence  little  ability  to  apply  it.  Students  who  had 
this  insight  have,  nevertheless,  continued  to  report  that  taking  the 
program  was  a  good  investment  of  time  for  them,  as  evidenced  by  the 
close  equivalence  of  original  and  follow-up  scores  on  variables  V7  and 
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Table  9 

Original  and  Follow-up  Means  of  Perceived 
Usefulness  of  STAT1 


Variable 

G76 

Follow-up 

G77 

Original 

Follow-up 

G78 

Original 

Follow-up 

V7 

5.4 

6.0 

5.3 

5.6 

5.2 

VI 9 

5.1 

5.8 

5.8 

5.5 

5.3 

V22 

3.6 

4.1 

4.5 

V25 

3.9 

3.4 

3.9 
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In  summary,  little  change  occurred  in  students’  perceptions  of 
the  STAT1  program  over  time.  Moderately  strong  liking  for  learning 
statistics  via  computer  and  the  sense  that  taking  the  program  was  a 
good  investment  of  their  time  have  prevailed  even  though  only  low 
moderate  applicability  was  reported.  Since  the  bulk  of  the  students 
participating  in  this  study  are  in  the  Master’s  program  and  may  not 
have  begun  a  thesis  yet,  this  seeming  lapse  in  consistency  may  alter 
in  time . 

Perceptions  of  Other  Aspects  of  STAT1 
Apprehension 

Subjects  were  asked  to  rate  their  apprehension  towards  taking 
Educational  Psychology  502  (EdPsy  502)  and  then  to  indicate  the 
reasons  for  this.  The  mean  level  of  apprehension  reported  for  the 
44  subjects  was  4.18  on  a  seven-point  scale.  The  main  reasons  for 
this  apprehension  were  related  to  dislike  for  mathematics,  doubt  about 
ability  to  do  statistics,  and  doubt  about  the  value  of  such  a  course. 

In  general,  it  was  found  that  persons  reporting  higher  levels  of 
apprehension  tended  to  have  less  previous  knowledge  of  statistics 
(r=-.43,  p<.01),  to  find  the  content  more  difficult  to  understand 
(r=-.53,  pc. 001),  and  to  rate  the  difficulty  level  of  STAT1  to  be 
higher  (r=-.60,  pc. 001)  than  persons  with  lower  ratings  of  apprehension. 
There  was  also  some  tendency  for  the  "high  apprehension"  people  to  rate 
the  material  as  less  interesting  (r=-.36,  pc. 01),  and  to  obtain  mod¬ 
erately  lower  final  course  grades  (r=-.34,  p<.05)  than  the  "low 
apprehension"  people.  The  program  apparently  does  not  provide  enough 
reassurance  to  reduce  anxiety  in  those  with  high  initial  apprehension. 
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The  initial  fears  and  apprehensions  may  prove  to  be  well-founded  for 
the  student  and  interfere  somewhat  with  learning,  thus  maintaining  or 
even  increasing  the  negative  feelings  toward  the  course  content. 

Difficulty  Level 

It  has  already  been  reported  that  more  apprehensive  students 
tended  to  rate  the  content  as  being  more  difficult.  The  mean  difficulty 
rating  for  the  44  subjects  was  4.39.  As  might  be  expected,  persons  who 
had  some  previous  knowledge  of  statistics  tended  to  find  the  content 
easier  to  understand  (r=.57,  pc. 001)  and  to  report  increased  liking 
for  taking  statistics  by  computer  (r=.33,  p< . 05) .  Since  self-pacing 
was  the  most  frequently  reported  advantage  of  STAT1,  it  is  reasonable 
to  presume  that  persons  with  previous  knowledge  of  statistics  preferred 
not  to  risk  being  held  back  by  students  with  no  previous  knowledge,  as 
might  occur  in  a  classroom  setting.  Reported  difficulty  of  content 
did  not  relate  very  highly  to  final  course  grade  (r=.26,  pc. 05)  which 
could  be  partially  a  function  of  the  low  variability  of  final  course 
grades,  although  the  general  difficulty  level  of  STAT1  did  correlate 
moderately  with  final  course  grades  (r=.41,  pc. 01) .  (Individual  pacing 
of  instruction  and  examinations  allows  students  to  be  assessed  when 
they  feel  ready,  resulting  in  less  variability  of  course  grades.) 

Research  Design 

The  number  of  lectures  on  research  design  received  by  each  of  the 
groups  varied,  with  G77  subjects  receiving  the  fewest.  The  research 
design  component  of  EdPsy  502  is  adjunctive  to  the  STAT1  program,  and 
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therefore  is  offered  only  in  lecture  style.  When  asked  to  indicate 
their  preference  for  having  research  design  taught  by  computer  in¬ 
struction,  the  mean  response  for  the  44  subjects  was  a  low  2.98  on  a 
seven-point  scale,  in  favor  of  learning  research  design  by  computer. 
Overall  the  subjects  would  prefer  not  to  learn  research  design  on 
the  computer.  This  coincides  with  a  disadvantage  of  learning  by 
computer  reported  by  101  of  the  subjects  as  "lack  of  opportunity  to 
interact  with  students  and  teaching  staff”. 

Several  persons  reported  wanting  more  research  design  included 
in  the  course.  This  seems  to  be  an  area  of  concern  and  possible  need 
for  change  since  those  who  wanted  more  research  design  classes  tended 
to  report  less  use  of  statistical  knowledge  in  doing  a  thesis  (r=-.47, 
p<.01).  With  the  bulk  of  statistical  calculation  being  done  by 
computers  using  prepackaged  programs,  the  burden  of  designing  a  study 
may  take  precedence  over  the  actual  application  of  computational 
techniques.  Another  consideration  is  that  many  students  reported 
feeling  uncomfortable  with  mathematical  principles  and  for  them, 
increased  emphasis  on  design  may  represent  the  opportunity  to  grapple 
with  research  successfully  in  a  non-mathematical  (i.e.  less  threatening) 
way. 

Use  of  Knowledge  Gained 

This  has  been  partially  discussed  previously  but  will  be  covered 
in  more  detail  here.  Subjects  were  asked  to  rate  the  amount  and  type 
of  use  they  had  made  or  were  making  of  the  knowledge  gained  in  EdPsy  502. 
The  type  of  use  was  divided  into  two  categories,  active  and  passive. 
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Active  use  was  defined  as  the  calculation  of  statistics  while  passive 
use  was  defined  as  reading  research  articles,  critiquing  the  work  of 
others,  or  advising  others.  Use  was  further  subdivided  into  thesis 
use  and  use  in  the  general  work  situation.  Thesis  use  of  the  knowledge 
gained  in  EdPsy  502  was  reported  to  be  somewhat  more  active  than  passive 
(total  thesis  use  with  active  use  r-.73,  p<.001;  total  thesis  use  with 
passive  use  r=.54,  p< . 01) .  General  use  of  knowledge  gained  tended  to 
be  somewhat  more  passive  than  active  (total  general  use  with  passive 
use  r=.71,  pc. 001;  total  general  use  with  active  use  r=.59,  pc. 001). 

Laboratory  assignments  can  provide  the  practical  experience  of 
handling  numbers  and  formulae,  helping  to  reinforce  applications  and 
promote  learning.  When  asked  to  rate  the  helpfulness  of  the  labs  which 
accompanied  STAT1,  subjects  gave  a  mean  rating  of  5.53.  As  subjects 
reported  labs  to  be  more  useful,  they  also  reported  increased  active 
use  of  the  material  learned  in  doing  a  thesis  (r=.40,  pc. 05).  This 
suggests  that  the  laboratory  assignments  are  fulfilling  a  valuable 
purpose  for  the  students  in  helping  them  to  analyze  their  own  research 
data. 

The  actual  amount  of  use  of  statistical  knowledge  in  either  an 
active  or  passive  role  was  reported  to  be  higher  by  students  who  achieved 
higher  final  grades  (r=.44,  pc. 01)  and  by  students  who  felt  that  taking 
STAT1  was  a  good  investment  of  their  time  (r=.46,  pc. 01)  than  by  students 
who  achieved  less  well.  While  indicating  encouraging  trends,  these 
figures  must  be  viewed  cautiously  as  they  include  ratings  for  only  30 
subjects.  The  remaining  14  subjects  did  not  rate  thesis  use,  stating 
that  they  had  not  yet  begun  a  thesis. 
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Other  CAI  Courses 

Although  few  other  full  length  university  level  courses  are  avail¬ 
able  on  computer  at  the  University  of  Alberta  (apart  from  Faculty  of 
Medicine  programs) ,  students  were  asked  whether  they  had  taken  any 
other  courses  via  CAI  and  whether  they  would  like  to  take  others.  Few 
students  had  had  any  other  experience  with  CAI  but  a  mean  rating  of  5.39 
indicated  that  most  were  in  favor  of  taking  other  courses  via  CAI.  As 
could  be  expected,  those  who  were  most  in  favor  of  taking  other  courses 
by  conputer  indicated  greater  liking  for  STAT1  (r=.55,  p<.001)  than 
those  who  were  less  in  favor  of  taking  other  courses  via  CAI,  reported 
that  taking  STAT1  was  a  good  investment  of  their  time  (r=.40,  p<.01), 
and  said  they  would  recommend  STAT1  to  others  (r=.35,  pc. 01). 

To  summarize,  students  who  were  less  apprehensive  about  EdPsy  502 
achieved  well  in  STAT1,  reported  more  positive  attitudes  toward  computer- 
assisted  instruction,  and  made  greater  use  of  the  material  learned  in 
the  program  than  those  who  were  more  apprehensive  at  the  outset.  The 
apprehensive  students  were  moderately  less  successful  in  the  course, 
expressed  less  positive  attitudes,  and  made  less  use  of  the  knowledge 
gained. 

Review  of  Research  Questions 

Several  research  questions  were  stated  in  Chapter  2  as  the  specific 
issues  to  be  addressed  in  this  study.  By  way  of  sumnary,  those  questions 
are  restated  and  addressed  below. 

1)  Did  achievement  scores  six  months  to  two  years  after  course  completion 

differ  from  those  gained  at  the  time  of  initial  testing? 


Scores  for  all  subjects  tended  to  be  about  201  lower  at  follow-up 
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testing  than  at  initial  testing.  This  experience  occurred  regardless 
of  the  time  span  between  testings. 

i 

2)  Did  retrospective  perceptions  of  the  STAT1  program  differ  from 
those  noted  at  the  time  of  initial  testing? 

Attitudes  toward  STAT1  and  learning  via  CAI  tended  to  remain  high 
for  all  subjects. 

3)  Did  the  amount  of  reported  use  of  statistics  have  any  relationship 
to  follow-up  perceptions  or  achievement  scores? 

There  was  little  relationship  between  the  amount  of  use  made  of 
statistics  and  the  achievement  scores  of  subjects  in  the  study.  There 
was,  however,  a  tendency  for  subjects  who  made  greater  use  of  the  know¬ 
ledge  gained  to  report  more  positive  attitudes  toward  STAT1. 

4)  What  were  the  specific  factors  which  tended  to  be  associated  with 
positive  and  negative  attitudes  toward  STAT1? 

Positive  attitudes  tended  to  be  associated  with  having  some  previous 
knowledge  of  statistics  prior  to  using  STAT1,  as  well  as  with  perceiving 
the  material  to  be  useful.  Negative  attitudes  tended  to  be  associated 
with  apprehension  about  taking  a  statistics  course,  lack  of  previous 
statistical  knowledge,  and  doubt  about  the  usefulness  of  the  material 
to  be  learned. 
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5)  How  helpful  were  features  outside  STAT1,  such  as  laboratory 
assignments  and  research  design  classes,  perceived  to  be? 

/ 

In  general,  students  reported  the  lab  assignments  to  be  very 
useful.  The  research  design  classes  were  frequently  isolated  as 
the  most  useful  component  of  the  EdPsy  502  course,  with  many  subjects 
stating  a  desire  for  more  classes  in  this  area. 

Subjects  also  offered  their  opinions  regarding  additions  and 
deletions  which  could  improve  the  STAT1  program.  These  were  discussed 
in  Chapter  5. 


Chapter  5 

Summary,  Recommendations  and  Conclusions 


Summary 

A  review  of  the  literature  on  the  use  and  evaluation  of  computer- 
assisted  instruction  in  education  revealed  a  paucity  of  detail  reported 
about  the  CAI  programs  in  question.  Details  such  as  length  of  time  to 
complete  the  program,  type  of  teaching  mode  employed,  and  the  particular 
purpose  of  each  program  (i.e.  supplementary  to,  or  replacement  for 
classroom  instruction)  were  most  often  lacking.  All  of  this  made  com¬ 
parison  of  findings  about  attitude  and  achievement  associated  with  CAI 
difficult,  if  not  impossible.  The  general  consensus  reported  by  the 
literature  regarding  the  state  of  CAI  at  present  was  that  it  serves  not 
as  a  replacement  to  classroom  instruction  but  as  a  supplement  to  it. 

Very  few  programs  seem  to  function  in  an  ongoing  manner  as  the  sole  mode 
of  instruction,  although  there  is  evidence  that  some  programs  have  been 
used  successfully  for  varying  periods  of  time. 

The  computer-assisted  instruction  facility  at  the  University  of 
Alberta  was  described  in  some  detail  with  attention  finally  focussed 
on  one  particular  program  called  STAT1.  This  program  takes  about  70 
hours  of  terminal  time  to  complete,  and  represents  the  bulk  of  instruc¬ 
tion  in  a  graduate  level  course  in  statistics  titled  Educational  Psychology 
502.  The  only  classroom  instruction  associated  with  the  program  con¬ 
sists  of  lecture/seminar  sessions  dealing  with  research  design  con¬ 
siderations.  These  sessions  supplement  the  STAT1  program,  rather  than 
vice  versa. 
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The  STAT1  program  has  been  functioning  since  the  summer  of  1975. 

While  continuing  ongoing  analysis  of  student  progress  has  been  carried 
out,  no  attempt  has  been  made  to  conduct  follow-up  comparisons  of 
attitude  or  achievement  since  the  inception  of  the  program. 

The  purpose  of  the  present  study  was  to  compare  follow-up  attitude- 
toward-CAI  ratings  and  achievement  scores  with  attitude  and  achievement 
measures  gathered  while  students  were  taking  the  program.  An  attempt 
was  also  made  to  gauge  the  amount  of  use  former  students  had  made  of 
the  statistical  knowledge  they  had  gained. 

Students  who  had  taken  the  course  during  the  winter  of  1976-77, 
the  summer  of  1977,  and  the  spring  and  summer  of  1978  were  identified 
as  subjects  for  the  study.  During  these  sessions  114  persons  had 
completed  the  program  but  the  whereabouts  of  only  93  people  were  known. 
Each  of  these  93  people  was  mailed  a  package  containing  a  15- item  achieve¬ 
ment  test,  a  31- item  opinionaire,  and  a  covering  letter  explaining  the 
purpose  of  the  study  and  requesting  their  cooperation  and  participation. 
The  achievement  tests  were  prepared  using  multiple  matrix  sampling 
procedures  so  that  each  subtest  would  be  short  enough  to  encourage  co¬ 
operation  of  the  subjects,  yet  allowing  estimates  of  population  para¬ 
meters  to  be  made.  In  all,  only  44  persons  participated  in  the  study, 
despite  additional  personal  contact  being  made  with  as  many  potential 
subjects  as  possible,  and  thus  estimating  confidence  intervals  for 
population  parameters  could  not  be  done. 

From  the  responses  of  these  44  subjects,  contrasts  were  described 
between  scores  on  the  15-item  achievement  test  and  the  final  course 
grade  received  by  each  subject,  as  well  as  between  ratings  on  opinionaire 
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items  at  follow-up  and  at  the  time  of  course  completion.  Only  39  of  the 
44  subjects  completed  the  achievement  test.  For  these  39  subjects,  the 
mean  score  was  57.79%  with  a  standard  deviation  of  18.27%.  The  mean 
of  the  final  course  grades  for  these  subjects  was  79.45%  with  a  standard 
deviation  of  10.74%.  The  Pearson  correlation  between  final  course 
grades  and  achievement  test  scores  was  .45  (p<. 01). 

Opinionaire  items  dealing  with  expressed  liking  for  CA.I  as  a 
teaching  modality,  and  with  the  use  made  of  the  material  studied,  were 
singled  out  for  additional  comparisons.  Pearson  correlations  were 
computed  between  various  aspects  of  expressed  liking  for  CAI  and  per¬ 
ceived  usefulness.  Certain  other  items,  such  as  those  dealing  with 
apprehension  about  taking  the  course,  usefulness  of  laboratory  assign¬ 
ments,  and  research  design  were  also  examined.  A  summary  of  the  results 
indicates  that  little  change  occurred  in  students’  perceptions  of  the 
STAT1  program  over  time.  Moderately  strong  liking  for  learning  stat¬ 
istics  via  computer  and  the  sense  that  taking  the  program  was  a  good 
investment  of  their  time  have  prevailed  even  though  only  low  moderate 
applicability  has  been  reported.  Students  who  were  less  apprehensive 
about  EdPsy  502  achieved  well  in  STAT1,  reported  more  positive  attitudes 
toward  computer- assisted  instruction,  and  made  greater  use  of  the 
material  learned  in  the  program  than  those  who  were  more  apprehensive 
at  the  outset. 

Because  the  overall  response  rate  was  considerably  lower  than 
anticipated,  a  brief  interview  was  conducted  with  all  those  who  initially 
agreed  verbally  to  participate  and  then  failed  to  do  so.  The  purpose 
of  the  interview  was  to  determine  which  factors  brought  about  the  change 
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of  mind.  The  most  common  problem  expressed  by  subjects  was  lack  of 
knowledge.  Most  felt  they  could  not  remember  enough  to  attempt  the 
achievement  test  without  doing  an  extensive  review  of  their  statistics 
notes.  Without  exception,  they  stated  that  they  could  not  possibly 
respond  to  the  15  items  in  the  45  minutes  suggested  on  the  question¬ 
naire.  When  this  topic  was  pursued  further,  many  revealed  not  only 
a  desire  to  do  well  on  the  test  (despite  the  guarantee  of  anonymity) , 
but  a  fear  of  not  doing  well  such  that  it  resulted  in  their  complete 
failure  to  attempt  the  items.  In  most  cases,  this  fear  of  failure  on 
the  test  did  not  translate  into  any  desire  to  upgrade  their  knowledge 
and  skills  in  statistics  but  rather  into  a  hesitancy  to  acknowledge 
what  they  already  felt  to  be  true.  Nearly  a  dozen  subjects  made  state¬ 
ments  along  this  theme:  "I  got  a  good  grade  in  the  course  but  I  know 
I've  forgotten  a  great  deal.  It's  just  kind  of  a  personal  thing,  feeling 
you  have  to  meet  the  standard  you  set  before,  and  not  wanting  to  try 
unless  you  know  you  can  do  it.".  This  reluctance  to  attempt  the 
achievement  questionnaire  accounts  for  the  discrepancy  between  the 
number  of  persons  responding  to  it  and  the  number  responding  to  the 
opinionaire,  since  five  persons  agreed  to  do  only  the  opinionaire. 

Recommendations  and  Conclusions 

The  purpose  of  this  study  was  to  conduct  a  follow-up  evaluation 
of  the  computer- assisted  instruction  program  called  STAT1,  designed  to 
teach  statistics  at  the  graduate  level  in  Educational  Psychology.  Those 
features  of  particular  interest  were  the  attitudes  of  the  students  to 
various  aspects  of  learning  by  computer,  and  an  estimate  of  learning 
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retention  over  time.  Ignoring,  for  the  moment,  attitudes  toward 
statistics  as  a  content  area,  it  seems  fairly  clear  that,  for  the 
most  part,  students  who  participated  in  the  course  liked  the  teaching 
medium  and  continue  to  hold  a  favorable  attitude  toward  CAI  six  months 
to  two  years  later.  When  making  recommendations,  however,  it  would  be 
unwise  to  ignore  attitudes  toward  the  content  of  the  course,  particularly 
as  the  mere  mention  of  statistics  seems  to  produce  a  highly  negative 
affect  in  many  people  who  take  the  course. 

In  the  process  of  collecting  the  data  for  this  study,  it  became 
increasingly  evident  that  most  people  had  taken  the  course  because  it 
was  required  as  part  of  their  graduate  program,  and  not  because  they 
recognized  the  value  of  understanding  statistics.  The  heavy  demand  for 
the  course  over  the  shorter  sessions  in  spring  and  summer  reinforce  this 
view.  Little  can  be  done  about  the  attitudes  people  develop  elsewhere 
toward  statistics  but  consideration  could  be  given  to  ways  in  which  the 
value  of  understanding  statistics  might  be  made  evident  to  those  taking 
the  program.  Adopting  an  attitude  which  reflects  recognition  of  the 
difficulties  encountered  by  those  who  consider  themselves  to  be  poor 
at  mathematical  conceptualization  is  a  first  step,  but  this  should  be 
accompanied  by  efforts  to  demonstrate  the  value  of  statistical  knowledge. 

Expand  the  course  length 

One  might  begin  by  expanding  the  time  available  for  the  completion 
of  the  program.  Students  all  have  their  reasons  for  taking  courses  in 
the  spring  and  summer  months  and  often  it  is  a  fast  way  to  "knock  off 
another  course".  Since  many  subjects  in  this  study  reported  that  there 
was  too  much  material  in  STAT1  to  be  covered  in  six  weeks,  perhaps 
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students  would  be  prepared  to  take  the  program  over  ten  or  twelve 
weeks  (e.g.  Spring  and  Summer  Sessions  combined)  while  still  realizing 
a  time  saving  over  regular  Winter  Session  programs  of  26  weeks. 

Expand  the  research  design  component 

A  frequent  request  made  by  subjects  was  that  more  time  be  spent 
on  research  design.  If  the  completion  time  were  expanded  from  six  to 
twelve  weeks,  some  additional  time  could  be  devoted  to  this  subject 
matter.  The  lecture/seminar  sessions  could  be  organized  to  correspond 
to  the  segments  of  STAT1  where  appropriate.  A  schedule  indicating  the 
correspondence  between  lectures  and  STAT1  would  encourage  students  to 
pace  themselves  so  that  the  appropriate  segments  were  completed  before 
the  lecture  component.  The  schedule  of  research  design  classes  given 
in  1978  indicates  that  only  one  2-hour  session  was  devoted  to  critiquing 
experiments.  Two  or  three  such  sessions  looking  at  published  studies 
and  the  students’  own  work  would  not  be  unreasonable.  Other  topics 
which  might  be  included  are  types  of  evaluation  studies,  alternative 
sampling  procedures,  and  the  use  of  multiple  baselines. 

Additions  to  STAT1 

Most  requests  for  changes  in  STAT1  came  in  the  form  of  additions 
to  the  program  such  as  more  concrete  examples  and  a  broader  range  of 
applicability.  If  such  additions  are  eventually  made,  it  could  also 
be  helpful  to  emphasize,  through  concrete  examples  and  comparisons,  when 
a  particular  statistical  test  should  be  chosen  over  another.  Although 
applicability  is  discussed  as  each  new  test  is  encountered,  nowhere 
are  the  various  statistical  tests  brought  together  so  that  the  dif¬ 
ferences  can  be  emphasized. 
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A  further  addition  to  STAT1  might  be  a  segment  consisting  of  a 
bank  of  research  problems  which  could  be  gleaned  from  published 
studies  and  used  as  practice  problems  to  give  the  student  experience 
in  designing  studies  to  answer  fairly  specific  questions.  Taken  in  a 
step-by-step  manner,  the  student  could  select  from  several  alternatives 
or  supply  his  own  responses  in  much  the  same  way  as  the  medical  students 
work  through  problems  of  patient  diagnosis  and  treatment.  Learning 
from  this  segment  could  then  be  augmented  by  having  students  submit 
written  critiques  of  research  articles. 

Laboratory  assignments  and  data  processing 

The  STAT1  segment  which  deals  with  processing  data  using  the 
computer  (segment  25)  was  ranked  by  subjects  as  being  one  of  the  most 
useful  in  helping  them  analyze  data  for  a  thesis.  Despite  the  recognized 
value  of  this  segment,  changes  could  be  made  to  make  it  easier  to  deal 
with.  Much  of  the  first  half  of  this  segment  is  text  with  no  problems 
or  questions  calling  for  a  response  from  the  student.  If  the  textual 
information  were  made  available  in  the  form  of  a  printed  handout,  students 
could  read  the  material  beforehand  and  come  to  the  CAI  terminal  prepared 
to  work  through  the  practical  problems  of  writing  format  statements, 
preparing  data  cards,  and  submitting  a  batch  job. 

Since  the  laboratory  assignments  were  generally  reported  to  be  a 
helpful  addition  to  STAT1,  several  sets  of  data  could  be  made  available 
which  would  be  suitable  for  analysis  using  the  different  computer  pro¬ 
grams  for  t-test,  ANOVA,  Chi  Square,  and  Pearson  r.  Most  data  analysis 
is  done  by  graduate  students  conducting  thesis  research  using 
these  prepackaged  computer  programs  but  little  opportunity  exists  to 
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practice  the  various  steps  required  to  prepare  the  data  cards  and  control 
cards  and  then  interpret  the  computer  printout  of  the  analysis.  A 
minimum  of  two  such  assignments  could  be  required  (instead  of  the  present 
one) ,  with  additional  ones  available  but  left  to  the  discretion  of  the 
individual  students. 

Other  suggestions 

Other  suggested  changes  in  STAT1  consist  mainly  of  housekeeping 
activities:  increase  the  use  of  titles  and  headings  in  segments  13,  16 
and  17,  improve  the  second  explanation  of  a  principle  where  one  is 
offered,  close  the  gap  between  sample  questions  and  test  questions,  and 
insert  restart  points  at  more  frequent  intervals  in  segments  4  and  25. 

As  it  stands,  STAT1  is  acceptable  to  most  students.  The  preceding 
suggestions  are  based  on  the  perceptions  of  people  who  have  had  first 
hand  experience  trying  to  learn  the  bulk  of  the  subject  matter  in  EdPsy 
502  via  computer- assisted  instruction.  As  such  the  suggestions  are 
intended  to  improve  the  program  for  future  use,  although  it  is  recog¬ 
nized  that  future  students  may  have  additional  ideas  for  improving  the 
program.  Since  the  type  of  comments  received  tended  to  be  fairly  con¬ 
sistent  across  groups,  one  may  be  fairly  certain  that  the  changes  re¬ 
quested  are  not  the  result  of  only  a  few  dissatisfied  students. 

The  Future  of  STAT1 

The  computer  equipment  which  is  currently  used  to  provide  computer- 
assisted  instruction  at  the  University  of  Alberta  has  been  designated 
obsolete  by  its  manufacturers  and  is  scheduled  for  removal  in  1980. 
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As  provisions  for  replacement  have  not  been  finalized,  there  will  likely 
be  a  period  of  time  during  which  no  CAI  programs  will  be  available  at 
the  university,  including  STAT1 .  Since  STAT1  has  shown  itself  to  be  a 
popular  and  reasonably  efficient  method  for  teaching  statistics,  one 
would  hope  that  provision  for  reinstating  this  program  will  not  be  too 
long  delayed. 

Despite  the  inconclusiveness  of  much  of  the  literature  pertaining 
to  attitude  and  achievement  studies  revolving  around  CAI ,  STAT1  has 
proven  to  be  regarded  positively  by  most  students  with  the  drawbacks 
being  far  outweighed  by  the  advantages.  Computer- ass is ted  instruction 
is  a  viable  alternative  to  classroom  instruction  and,  while  the  results 
of  this  study  are  not  intended  to  indicate  that  students  prefer  CAI 
over  classroom  instruction  in  all  instances,  they  certainly  suggest 
that  students  prefer  to  learn  statistics  via  CAI. 
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End  of  Segment  Opinionaire  (STAT1) 


1.  I  found  the  material  in  this  chapter  was  quite  useful  in  helping 
me  learn  basic  statistics. 

2.  The  material  in  this  chapter  was  quite  boring. 

3.  I  found  this  chapter  easy  to  understand. 

4 .  I  previously  had  not  been  taught  any  of  the  material  in  this 

chapter . 

5 .  There  was  never  anyone  available  in  the  terminal  room  to  help 
answer  my  questions  about  this  chapter. 

6.  I  feel  that  someone  who  is  knowledgeable  in  the  course  content 
of  this  chapter  should  always  be  available  in  the  terminal  room. 

7.  Whenever  there  was  a  course  content  specialist  available  in  the 
terminal  room  that  person  very  often  did  not  know  the  material 
very  well . 

8.  I  feel  that  it  would  have  been  easier  for  me  to  learn  the  material 
in  this  chapter  by  classroom  instruction  rather  than  by  computer- 
assisted  instruction. 
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APPENDICES 

STAT1  OPINIONAIRE 


Please  answer  the  following  questions  about  your  perceptions  of 
the  STAT1  computer- assisted  instruction  program  as  it  was  used  in 
Educational  Psychology  502. 

Indicate  your  response  by  placing  an  "X"  in  the  a-propriate  space. 


1.  In  which  session  did  you  take  Ed.  Psych.  502  via  computer? 


Winter  1976-77  _  Spring  1977  _ 

Spring  1978  _  Summer  1978  _ 

2.  At  what  point  in  your  program  did  you  take  Ed.  Psych.  502? 


1st  year  _  2nd  year  _  Other  (Explain) 


3.  Rate  the  importance  of  each  factor  in  your  decision  to  take  Ed.  Psych 
502  when  you  did. 


Not 

Imp. 

a)  computer- assisted 

instruction  course  _ 

b)  positive  comments 
from  other  students 


c)  the  instructors 

d)  concerned  about 
going  at  my  own 
speed 

e)  other  (specify) 


4.  I  was  apprehensive  about  taking  Ed.  Psych  502. 


Very 

Imp. 


Strongly 

Agree 


Strongly 

Disagree 


. . . Opinionaire 


2. 


5.  My  apprehension  was  based  on 
_  dislike  for  mathematics 

_  uncertainty  about  taking  a  course  via  computer 

_  doubt  about  the  usefulness  of  a  statistics  course 

_  other  (please  specify) 


6. 

I  found  the  STAT1  program  was  useful  in  helping  me  learn 

statistics . 

Strongly 

Strongly 

Agree 

Disagree 

7. 

It  would  have  been  easier  to  learn  the  STAT1  material  by  classroom 

instruction  rather  than  by  computer  instruction. 

Strongly 

Strongly 

Agree 

Disagree 

8. 

The  material  in  the  program  was: 

Very 

Very 

Boring 

Interesting 

9. 

Generally  the  content  was  easy  to  understand. 

Strongly 

Strongly 

Agree 

Disagree 

10. 

I  would  rate  the  difficulty  level  of  STAT1  as  being: 

Very 

Very 

Easy 

Difficult 

11. 

What  portion  of  the  material  covered  in  STAT1  had  you  previously  been 

taught? 

None 

All 

12.  Is  there  anything  which  could  be  added  to  improve  STAT1? 


’ 
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. .  .Opinionaire _ _ _ 3. 

13.  Are  there  any  sections  of  STAT1  which  could  be  dropped? 


14.  How  helpful  were  the  labs  which  accompanied  STAT1? 

Very  Not 

Helpful  At  All 


15.  I  would  have  preferred  to  have  the  research  design  component  taught 
by  computer  rather  than  in  seminar. 

Strongly  Strongly 

Agree  Disagree 


16.  I  would  like  to  have  had  more  seminar  sessions  scheduled. 

Strongly  Strongly 

Agree  Disagree 


17.  It  was  useful  to  have  the  computer  terminals  available  on  Saturdays 
and  Sundays . 

Always  Never 


18.  Someone  who  is  knowledgeable  of  the  STAT1  content  should  be  avail¬ 
able  in  the  terminal  room  to  answer  questions. 

Strongly 
Agree 


19.  I  liked  taking  statistics  via  computer -as sis ted  instruction. 

Strongly  Strongly 

Agree  Disagree 


20.  Taking  statistics  via  computer- ass is ted  instruction  was  a  good  in¬ 
vestment  of  my  time. 


Strongly 

Disagree 


Strongly 

Agree 


Strongly 

Disagree 
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. . .Opinionaire 


4. 


21.  I  would  recommend  the  STAT1  program  to  other  graduate  students 
or  educational  personnel. 


Strongly 

Agree 


Strongly 

Disagree 


22.  What  do  you  think  were  the  major  advantages  of  using  the  STAT1 
program  as  part  of  Ed.  Psych.  502? 


23.  What  do  you  think  were  the  major  disadvantages  of  using  the  STAT1 
program  as  part  of  Ed.  Psych  502? 


24.  I  would  like  to  take  other  courses  via  computer- assisted  instruction. 

Strongly  Strongly 

Agree  Disagree 


25.  Have  you  taken  other  courses  via  computer- assisted  instruction? 

No  _  Yes  _  (Please  give  details) 

26 .  Are  there  any  subsections  of  the  STAT1  program  which  have  been 
particularly  useful?  Please  list. 


27.  Which  components  learned  in  STAT1  have  you  used  since  you  took  the 
program? 


. 


ft  i/0 


.  .Opinionaire 


5. 


28.  How  useful  was  (is)  the  material  learned  in  Ed.  Psych.  502  in  doing 
your  thesis? 

Very  Not 

Useful  At  All 


29.  In  doing  my  thesis  I  used  (am  using)  the  material  learned  in  Ed. 
Psych.  502  in: 

a)  an  active  role,  i.e.,  to  calculate  statistics 


Very 

Much 


Not 
At  All 


b)  a  passive  role,  i.e.,  to  read  journals  and  critique  the  work  of 
others,  to  advise  others 

Very  Not 

Much  At  All 


30.  How  useful  was  (is)  the  material  learned  in  Ed.  Psych.  502  in  your 
general  work  situation? 


Very 

Useful 


Not 
At  All 


31.  In  my  general  work  situation  I  use  the  material  learned  in  Ed.  Psych. 
502  in: 

a)  an  active  role 

Very  Not 

Much  '  At  All 


b)  a  passive  role 

Very 

Much 


Not 

At  All 


. 
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January  3,  1979 


Dear 


As  you  are  probably  aware,  Ed.  Psy.  502  (statistics)  has  been 
offered  via  computer- as sis ted  instruction  for  several  years  now. 
Although  the  program  seems  to  work  well  and  students  offer  positive 
feedback,  follow-up  evaluation  of  a  formal  nature  is  needed  to 
determine  program  effectiveness  over  time. 

Therefore,  course  graduates  like  yourself  are  being  asked  to 
donate  30  to  45  minutes  to  aid  in  this  evaluation  by  completing 
1)  an  opinionaire  much  like  the  one  you  filled  out  at  the  end  of  the 
course,  and  2)  an  achievement  questionnaire  which  samples  the  course 
material.  Copies  are  enclosed. 

Please  answer  the  opinionaire  first .  It  will  take  only  2  or  3 
minutes  to  complete.  Then  answer  the  15  item  achievement  question¬ 
naire  which  should  take  about  30  minutes .  Please  do  not  spend  more 
than  45  minutes  on  it.  You  may  do  it  as  an  open  book  exam  but  work 
independently.  The  results  will  not  identify  you  as  an  individual, 
nor  will  they  be  used  for  any  purpose  other  than  this  evaluation. 

Please  return  the  questionnaires  in  the  envelope  provided,  by 
January  22,  1979.  If  you  have  any  questions,  please  feel  free  to 
contact  me  through  the  Ed.  Psych.  Dept,  at  the  university. 

Thank  you  for  your  time  and  cooperation. 

Yours  truly, 


Fran  Vargo 
Graduate  Student 


